Video imaging an area of interest using networked cameras

ABSTRACT

The systems, methods, and/or computer-readable media described herein allow a reviewer to review video content of multiple perspectives of area(s) of interest at a specific time using a system of networked and time-synchronized video cameras. The networked and time-synchronized video cameras may comprise dedicated video cameras or may be coupled to mobile phones or tablet computing devices, and/or those incorporated in action housings. The networked and time-synchronized video cameras may capture multiple perspectives of area(s) of interest in that they may be arranged so that their fields of view are directed toward different orientations with respect to the area(s) of interest.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/773,123 filed May 2, 2018, now U.S. Pat. No. 10,595,008,which is a national stage entry of International Application No.PCT/US2016/059783 filed Oct. 31, 2016, which claims priority to U.S.Provisional Patent Application Ser. No. 62/248,066 filed Oct. 29, 2015,U.S. Provisional Patent Application Ser. No. 62/345,696 filed Jun. 3,2016, and U.S. Provisional Patent Application Ser. No. 62/381,261 filedAug. 30, 2016, all of which are incorporated by reference herein.

BACKGROUND

Cameras have formed a part of coaching and other review tools. As anexample, video cameras have been used to capture video content ofsports, dance routines, and other activities. People may use the videocontent to review, either alone or with a coach, teacher, or otherprofessional, their situational approaches, tactics, techniques, etc. Asanother example, police and/or security personnel may use video camerasto capture and review video content captured from security camerasand/or during investigations.

It may be desirable for a reviewer to view multiple perspectives of aspecific area at a specific time. A coach or dance teacher, forinstance, may find it useful to view multiple angles of a specificaction or routine taken by a player or a student. A police officer orsecurity personnel may find it useful to view a suspect from multipleangles at a specific time to assess the suspect's credibility, demeanor,etc. In many instances, however, video cameras may be limited tocapturing only the items within their field of view, and therefore, onlyone perspective. It may be desirable to technically simplify capture ofmultiple perspectives of an area of interest at a specific time withoutimplementing complicated processing steps after video capture orrequiring a reviewer to watch in parallel multiple video feeds of anarea of interest.

SUMMARY

The systems, methods, and/or computer-readable media described hereinallow a reviewer to review video content of multiple perspectives ofarea(s) of interest at a time using a system of networked andtime-synchronized video cameras. The networked and time-synchronizedvideo cameras may comprise dedicated video cameras or may be coupled tomobile phones or tablet computing devices, and/or those incorporated inaction housings. The networked and time-synchronized video cameras maycapture multiple perspectives of area(s) of interest in that they may bearranged so that their fields of view are directed toward differentorientations with respect to the area(s) of interest.

The networked and time-synchronized video cameras may betime-synchronized in that they begin capturing video content related tothe area(s) of interest at approximately the same time. The videocontent from the networked and time-synchronized video cameras may beused to form a three-dimensional dome representation of the area(s) ofinterest, which, as described further herein, may allow a reviewer toview video content of any of the perspectives of the area(s) of interestat a specific time. The three-dimensional dome representation of thearea(s) of interest may also include one or more orientation markers,which, as described further herein, may allow a reviewer to switchbetween perspectives of the area(s) of interest at a specific time. Aplayback device associated with the reviewer may be configured todisplay a stitched video representation of the area(s) of interest thatis based on the three-dimensional dome representation. The stitchedvideo representation may include one or more perspective user interface(UI) elements corresponding to the orientation markers that allow thereviewer to switch between video perspectives of the area(s) of interestat a specific time using the graphical interface of the playback device.

These and other advantages will become apparent to those skilled in therelevant art upon a reading of the following descriptions and a study ofthe several examples of the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A depicts a diagram of an example of a time-synchronized videocapture environment.

FIG. 1B depicts a diagram of an example of a time-synchronized videocapture environment.

FIG. 1C depicts a diagram of an example of a time-synchronized videocapture environment.

FIG. 1D depicts a diagram of an example of a time-synchronized videocapture environment.

FIG. 2 depicts a diagram of an example of a time-synchronized videocapture device.

FIG. 3 depicts a diagram of an example of a playback device.

FIG. 4 depicts a diagram of an example of a time-synchronized videocapture management system.

FIG. 5 depicts a flowchart of an example of a method for capturingtime-synchronized video content of a visible portion of physical area(s)of interest.

FIG. 6 depicts a flowchart of an example of a method for incorporatingvideo content of a plurality of perspectives of one or more areas ofinterest into a three-dimensional dome representation of the one or moreareas of interest.

FIG. 7 depicts a flowchart of an example of a method for displaying astitched video representation of one or more areas of interest on aplayback device.

FIG. 8A shows an example of a screenshot of a review application on aplayback device.

FIG. 8B shows an example of a screenshot of a review application on aplayback device.

FIG. 8C shows an example of a screenshot of a review application on aplayback device.

FIG. 8D shows an example of a screenshot of a review application on aplayback device.

FIG. 8E shows an example of a screenshot of a review application on aplayback device.

FIG. 9 depicts a diagram of an example of a computer system.

FIG. 10 is a schematic illustration of an environment.

FIG. 11 is an example illustration depicting orientation arrangements ofa plurality of cameras around an object.

FIG. 12 is an example illustration depicting orientation arrangements ofa plurality of cameras around an object.

FIG. 13 is an example illustration for determining relative positions ofthe plurality of cameras having the orientation arrangements of FIGS.2-3.

FIG. 14 is an example illustration for determining relative positions ofa plurality of cameras and order thereof.

FIG. 15 is a schematic illustration of a camera and its orientationinformation with respect to a co-ordinate system.

FIG. 16 is schematic illustration of a plurality of cameras and itsorientation information with respect to another coordinate system.

FIG. 17 depicts a flowchart of an example of a method for determiningrelative positions of the plurality of cameras with respect to eachother within a location.

DETAILED DESCRIPTION Examples of Time-Synchronized Video CaptureEnvironments

FIG. 1A depicts a diagram 100A of an example of a time-synchronizedvideo capture environment. The diagram 100A includes a computer-readablemedium 102, time-synchronized video capture devices 104, physicalareas(s) of interest 106, a time-synchronized video capture managementsystem 108, and playback device(s) 110. In the diagram 100A, thecomputer-readable medium 102 is coupled to the time-synchronized videocapture devices 104, the time-synchronized video capture managementsystem 108, and the playback device(s) 110.

The computer-readable medium 102 and other computer readable mediumsdiscussed in this paper are intended to represent a variety ofpotentially applicable technologies. For example, the computer-readablemedium 102 can be used to form a network or part of a network. Where twocomponents are co-located on a device, the computer-readable medium 102can include a bus or other data conduit or plane. Where a firstcomponent is co-located on one device and a second component is locatedon a different device, the computer-readable medium 102 can include awireless or wired back-end network or local area network (LAN). Thecomputer-readable medium 102 can also encompass a relevant portion of awide area network (WAN), such as the Internet, or other network, ifapplicable.

The computer-readable medium 102 and other applicable systems or devicesdescribed in this paper can be implemented as a computer system or partsof a computer system or a plurality of computer systems. In general, acomputer system will include a processor, memory, non-volatile storage,and an interface. A typical computer system will usually include atleast a processor, memory, and a device (e.g., a bus) coupling thememory to the processor. The processor can be, for example, ageneral-purpose central processing unit (CPU), such as a microprocessor,or a special-purpose processor, such as a microcontroller.

The memory can include, by way of example but not limitation, randomaccess memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM).The memory can be local, remote, or distributed. The bus can also couplethe processor to non-volatile storage. The non-volatile storage is oftena magnetic floppy or hard disk, a magnetic-optical disk, an opticaldisk, a read-only memory (ROM), such as a CD-ROM, EPROM, or EEPROM, amagnetic or optical card, or another form of storage for large amountsof data. Some of this data is often written, by a direct memory accessprocess, into memory during execution of software on the computersystem. The non-volatile storage can be local, remote, or distributed.The non-volatile storage is optional because systems can be created withall applicable data available in memory.

Software is typically stored in the non-volatile storage. Indeed, forlarge programs, it may not even be possible to store the entire programin the memory. Nevertheless, it should be understood that for softwareto run, if necessary, it is moved to a computer-readable locationappropriate for processing, and for illustrative purposes, that locationis referred to as the memory in this paper. Even when software is movedto the memory for execution, the processor will typically make use ofhardware registers to store values associated with the software, andlocal cache that, ideally, serves to speed up execution. As used herein,a software program is assumed to be stored at an applicable known orconvenient location (from non-volatile storage to hardware registers)when the software program is referred to as “implemented in acomputer-readable storage medium.” A processor is considered to be“configured to execute a program” when at least one value associatedwith the program is stored in a register readable by the processor.

In one example of operation, a computer system can be controlled byoperating system software, which is a software program that includes afile management system, such as a disk operating system. One example ofoperating system software with associated file management systemsoftware is the family of operating systems known as Windows® fromMicrosoft Corporation of Redmond, Wash., and their associated filemanagement systems. Another example of operating system software withits associated file management system software is the Linux operatingsystem and its associated file management system. The file managementsystem is typically stored in the non-volatile storage and causes theprocessor to execute the various acts required by the operating systemto input and output data and to store data in the memory, includingstoring files on the non-volatile storage.

The bus can also couple the processor to the interface. The interfacecan include one or more input and/or output (I/O) devices. Dependingupon implementation-specific or other considerations, the I/O devicescan include, by way of example but not limitation, a keyboard, a mouseor other pointing device, disk drives, printers, a scanner, and otherI/O devices, including a display device. The display device can include,by way of example but not limitation, a cathode ray tube (CRT), liquidcrystal display (LCD), or some other applicable known or convenientdisplay device. The interface can include one or more of a modem ornetwork interface. It will be appreciated that a modem or networkinterface can be considered to be part of the computer system. Theinterface can include an analog modem, ISDN modem, cable modem, tokenring interface, satellite transmission interface (e.g. “direct PC”), orother interfaces for coupling a computer system to other computersystems. Interfaces enable computer systems and other devices to becoupled together in a network.

The computer systems can be compatible with or implemented as part of orthrough a cloud-based computing system. As used in this paper, acloud-based computing system is a system that provides virtualizedcomputing resources, software and/or information to end user devices.The computing resources, software and/or information can be virtualizedby maintaining centralized services and resources that the edge devicescan access over a communication interface, such as a network. “Cloud”may be a marketing term and for the purposes of this paper can includeany of the networks described herein. The cloud-based computing systemcan involve a subscription for services or use a utility pricing model.Users can access the protocols of the cloud-based computing systemthrough a web browser or other container application located on theirend user device.

A computer system can be implemented as an engine, as part of an engineor through multiple engines. As used in this paper, an engine includesone or more processors or a portion thereof. A portion of one or moreprocessors can include some portion of hardware less than all of thehardware comprising any given one or more processors, such as a subsetof registers, the portion of the processor dedicated to one or morethreads of a multi-threaded processor, a time slice during which theprocessor is wholly or partially dedicated to carrying out part of theengine's functionality, or the like. As such, a first engine and asecond engine can have one or more dedicated processors or a firstengine and a second engine can share one or more processors with oneanother or other engines. Depending upon implementation-specific orother considerations, an engine can be centralized or its functionalitydistributed. An engine can include hardware, firmware, or softwareembodied in a computer-readable medium for execution by the processor.The processor transforms data into new data using implemented datastructures and methods, such as is described with reference to the FIGS.in this paper.

The engines described in this paper, or the engines through which thesystems and devices described in this paper can be implemented, can becloud-based engines. As used in this paper, a cloud-based engine is anengine that can run applications and/or functionalities using acloud-based computing system. All or portions of the applications and/orfunctionalities can be distributed across multiple computing devices,and need not be restricted to only one computing device. In someembodiments, the cloud-based engines can execute functionalities and/ormodules that end users access through a web browser or containerapplication without having the functionalities and/or modules installedlocally on the end-users' computing devices.

As used in this paper, datastores are intended to include repositorieshaving any applicable organization of data, including tables,comma-separated values (CSV) files, traditional databases (e.g., SQL),or other applicable known or convenient organizational formats.Datastores can be implemented, for example, as software embodied in aphysical computer-readable medium on a specific-purpose machine, infirmware, in hardware, in a combination thereof, or in an applicableknown or convenient device or system. Datastore-associated components,such as database interfaces, can be considered “part of” a datastore,part of some other system component, or a combination thereof, thoughthe physical location and other characteristics of datastore-associatedcomponents is not critical for an understanding of the techniquesdescribed in this paper.

Datastores can include data structures. As used in this paper, a datastructure is associated with a particular way of storing and organizingdata in a computer so that it can be used efficiently within a givencontext. Data structures are generally based on the ability of acomputer to fetch and store data at any place in its memory, specifiedby an address, a bit string that can be itself stored in memory andmanipulated by the program. Thus, some data structures are based oncomputing the addresses of data items with arithmetic operations; whileother data structures are based on storing addresses of data itemswithin the structure itself. Many data structures use both principles,sometimes combined in non-trivial ways. The implementation of a datastructure usually entails writing a set of procedures that create andmanipulate instances of that structure. The datastores, described inthis paper, can be cloud-based datastores. A cloud-based datastore is adatastore that is compatible with cloud-based computing systems andengines.

In the example of FIG. 1A, the time-synchronized video capture devices104 are intended to represent a plurality of devices configured tocapture video content at the same time (i.e., in a time-synchronizedmanner). “Video content,” as used herein, may refer to a series ofimages of an area of interest taken over a specified time. Dependingupon implementation- and/or configuration-specific considerations, videocontent can include corresponding audio content and, in applicableinstances, the time-synchronized video capture devices 104 can insteadbe referred to as time-synchronized multimedia capture device(s). It maybe noted audio capture device(s) need not be incorporated into the samedevice as the time-synchronized video capture devices 104, and can beimplemented as separate devices (not shown) coupled to thecomputer-readable medium 102. It may further be noted some techniquesdescribed in this paper are applicable to time-synchronized imagecapture devices that take (or can be configured to take) snapshots.

The time-synchronized video capture devices 104 may comprise engines,datastores, and/or other components of a computer system. For example,the time-synchronized video capture devices 104 can be coupled to orform a part of smartphones, tablet computers, Internet of Things (IoT)devices, or the like. The time-synchronized video capture devices 104may be coupled to and/or part of an unmanned vehicle(s), such as anunmanned aerial vehicle and/or drone.

In various implementations, the time-synchronized video capture devices104 include sensors other than cameras for capturing video. For example,the time-synchronized video capture devices 104 can include orientation-or location-sensitive sensors, such as a gyroscope, an accelerometer, aGPS sensor, a magnetometer, and/or a rangefinder. The time-synchronizedvideo capture devices 104 can also include sound sensors, such asmicrophones or ultrasonic sensors, alternative light sensors, such aslight sensors that capture images outside of the visible light range,thermometers or thermocouples, pressure or flow sensors, potentiometersand force-sensing resistors, humidity sensors, gas sensors, PIR motionsensors, acceleration sensors, displacement sensors, force measurementsensors, color sensors, gyro sensors, and other sensors.

For sensors that detect stimuli traveling at different speeds, sensordistance is of critical importance to determine an offset whencorrelating relatively slow-moving stimuli (such as sound) withrelatively fast-moving stimuli (such as light). As used here,“relatively” refers to the stimuli being compared (e.g., sound andlight, respectively). The offset is optional in the sense it may bedesirable to have a realistic time-delay for sound, such as whenobserving a batter hit a ball and hearing the crack of the impact of batto ball shortly thereafter. However, as the physical area(s) of interestincrease in size, the different sounds at the video capture devicesbecome increasingly disconcerting. In a specific implementation, thesound recording of only one of the video capture devices 104 is used. Inan alternative implementation, not shown in the figures, one or morededicated sound recording devices can be placed in close proximity tothe source of the sound. As used here, close proximity means closer tothe source of sound than at least one video capture device for thepurpose of capturing sound with a smaller offset than would be necessaryif the sound was received at the at least one video capture device.

An advantage of using time-synchronized devices is frames can be alignedfor various perspectives without substantial pre-processing. Introducingoffsets for sound can increase the amount of pre-processing, but, in atleast some implementations, minimizing the pre-processing is desirable.A technique for reducing pre-processing requirements for sound offsetsinvolves providing a tagging mechanism for objects in the physical areasof interest. For example, baseball players are routinely given articlesthat facilitate easy identification of the runners (such as uniformswith numbers on them). The visual identification techniques can beaugmented by providing a microphone in association with the identifyingarticle, which may or may not also include a passive or active componentused for determining the present location of the object and/or themicrophone. When the object is viewed (including zooming in on theobject from a larger composite image), the sound feed can be correlatedto the appropriate frame of the video feed by matching a sound inspace-time with the applicable frame.

In a specific implementation, a sound-recording device istime-synchronized with the time-synchronized video capture devices 104using a sound offset. That is, the time-synchronized video capturedevices 104 operate at a slight delay relative to the sound recordingdevice. In the most basic implementation, a sound recording device isplaced at a distance from a focal point and a video recording device isplaced at a second distance from the focal point. For simplicity, it canbe assumed the time it takes for light to travel from the focal point tothe second distance is 0 because the frame rate of video recordingdevices, even those operating at high frame rates (e.g., 240 FPS) cannotdetect a time-delay between a first object that is near and a secondobject that is all the way on the other side of, for example, a largearena. However, the time it takes for sound to travel from first andsecond objects on opposite sides of an arena (one near and one far) tothe video recording device can be significantly different, as isdemonstrated when a baseball player hits a ball and the crack is heardimmediately by the catcher, but moments later by fans in the stands. Tooffset this phenomenon, video can be offset by d/c, where d is thedistance of the recording device from the focal point and c is the speedof sound.

In the example of FIG. 1A, the physical area(s) of interest 106 areintended to represent a physical space that can be imaged by one or moreof the time-synchronized video capture devices 104. The physical area(s)of interest 106 may correspond to a relevant space around a person, suchas an athlete, a performer, or other person whose actions are recordedby the time-synchronized video capture devices 104. In someimplementations, the physical area(s) of interest 106 comprise a singlephysical area of interest that corresponds to the boundaries around aspecific person being recorded at a specific time by thetime-synchronized video capture devices 104. In various implementations,the physical area(s) of interest 106 comprise multiple physical areas ofinterest that correspond to boundaries around a single person beingrecorded at different times by the time-synchronized video capturedevices 104. In various implementations, the physical area(s) ofinterest 106 comprise multiple physical areas of interest thatcorrespond to boundaries around one or more persons being recorded atdifferent times by the time-synchronized video capture devices 104.

In the example of FIG. 1A, the physical area(s) of interest 106 havevisible portions 114 that are intended to represent conceptual windowsinto the physical area(s) of interest 106 through which thetime-synchronized video capture devices 104 capture images. As isillustrated in the example of FIG. 1A, the visible portions 114 of thephysical area(s) of interest 106 correspond to the fields of view 116 ofthe time-synchronized video capture devices 104. Thus, thetime-synchronized video capture devices 104 can be characterized by afield of view 116 that corresponds to the extent of the observable worldthat is seen at any given moment (or over a period of time) by thetime-synchronized video capture devices 104 that is associated with thephysical area(s) of interest 106. In the example of FIG. 1A, the fieldof view 116 is directed toward the visible portions 114 of the physicalarea(s) of interest 106. In this example, the field of view 116encompasses no more than the visible portions 114, but it should beunderstood the field of view could encompass more than just the visibleportions 114 of the physical area(s) of interest 106, making portions ofthe captured images extraneous. It may be desirable to avoid extraneousportions of images by orienting the time-synchronized video capturedevices 104 appropriately. Alternatively or in addition, extraneousportions of images can be clipped to yield only the visible portions 114of the physical area(s) of interest 106.

The visible portions 114 may, but need not, depend on the orientationsof the time-synchronized video capture devices 104. As an example, thevisible portions 114 may overlap with one another (e.g., the firstvisible portion 114(1) may overlap with the Nth visible portion 114(N)).As another example, the visible portions 114 may comprise perspectivesof the physical area(s) of interest 106 that are at least in partorthogonal to one another (e.g., the first visible portion 114(1) maycomprise a top-down perspective of the physical area(s) of interest 106while the Nth visible portion 114(N) may comprise a lateral perspectiveof the physical area(s) of interest 106). As yet another example, thefirst visible portion 114(1) may comprise a first lateral of thephysical area(s) of interest 106 while the Nth visible portion 114(N)may comprise a second lateral perspective of the physical area(s) ofinterest 106. It is noted other combinations of perspectives arepossible without departing from the scope and substance of the inventiveconcepts described herein.

In a specific implementation, the time-synchronized video capturemanagement system 108 can use sensors, such as a thermometer and/or ahumidity detector, to estimate the speed of sound in a givenenvironment. The speed of sound, c, varies depending upon airtemperature, humidity, and other factors. For example, if the sensors(or a weather report) indicate the temperature is 0 C with 0% humidityat sea level, c can be estimated to be about 331.3 m/s. Assuming a focalpoint 331 meters from a video recording device operating at 120 FPS, thesound offset is 120 frames. That is, the sound that is heard one secondafter an event is applicable to a frame 120 frames before the videorecording device detected the sound. Moreover, the applicable frame isdeterminable at any point between the focal point and some other videorecording device for which the distance is known using the formula−FPS×d/c. Specifically in the above example, the applicable frame offsetis −120 frames/s×(331 m)/(331 m/s)=−120 frames.

A reason for ensuring video recording devices are time-synchronized isto ensure the frames line up on feeds from each device. If two recordingdevices record at 120 FPS, but one starts recording 1/240 of a secondbefore the other, the frames are mismatched. If mismatched frames arestitched together, the visual experience is diminished by introducing a“jerkiness” to the video feed when perspective changes from one camerato the next. Sounds must also be carefully time-synchronized with videoto avoid, for example, having sounds not perfectly correspond to imageswithin frames. To this end, the start time of a sound recording ispinned to the start of the first frame and is at least conceptuallybroken into segments of a duration equal to 1/FPS. Thus, for a 120 FPSfeed, the sound segments are each 1/120^(th) of a second long and thestart of any given segment is pinned to the start of the correspondingframe. Advantageously, pinning sound segments to frames requiresrelatively little consumption of compute, enabling the pinning to beaccomplished in real-time with a relatively small delay, perhaps around3 seconds for a feed that includes a focal point about 1 km from theapplicable sensor.

In a specific implementation, the time-synchronized video capturedevices 104 are arranged at different orientations around the physicalarea(s) of interest 106 in order to capture different perspectives ofthe physical area(s) of interest 106. As an example, thetime-synchronized video capture devices 104 may be oriented around thephysical area(s) of interest 106 so that a portion of the first field ofview 116(1) of the first time-synchronized video capture devices 104(1)overlaps with a portion of the Nth field of view 116(N) of the Nthtime-synchronized video capture devices 104(N). As another example, thetime-synchronized video capture devices 104 may be oriented around thephysical area(s) of interest 106 so that a portion of the first field ofview 116(1) of the first time-synchronized video capture devices 104(1)is orthogonal with a portion of the Nth field of view 116(N) of the Nthtime-synchronized video capture devices 104(N).

In some implementations, the time-synchronized video capture devices104, when arranged, are mounted on one or more stands or frames and/orfacilitate video capture of the physical area(s) of interest 106 fromvarious perspectives. The one or more stands or frames can be configuredinto an arbitrary or non-arbitrary shape, such as a dome, sphere,hemisphere, cylinder, oval, line, plane, cube, or other shape. In someimplementations, the spaces at which the time-synchronized video capturedevices 104 are to be placed can be pre-marked, such as by putting an‘x’ at locations around a potential area of interest. Moreover, if thepositions are pre-plotted, it makes determination of locationsrelatively trivial for other components with a need to know the variouslocations.

In a specific implementation, a portion of the time-synchronized videocapture devices 104 is configured to move with an object in the physicalarea(s) of interest 106. For instance, the time-synchronized videocapture devices 104 may be mounted on a platform that moves along withan object in the physical area(s) of interest. Instead or in addition,the time-synchronized video capture devices 104 can change the field(s)of view 116 to accommodate an object moving in the physical area(s) ofinterest 106. For example, the time-synchronized video capture devices104 may be configured to rotate around a base to follow an object withinthe physical area(s) of interest 106. In a specific implementation, thetime-synchronized video capture devices 104 and/or the field(s) of view116 follow an object using an object (a fob, a beacon, etc.) on theobject in the physical area(s) of interest 106. In general, it is lessimportant for omnidirectional sensors, such as microphones, to move withan object, though it can be important to track distance from the objectfor the purpose of computing, e.g., a time-variable sound offset, or forswitching from a first microphone to a second microphone as an objecttraverses a path, such as a racetrack or bases on a baseball field.

In a specific implementation, the time-synchronized video capturedevices 104 are arranged according to a Cartesian coordinate systemand/or substantially Cartesian coordinate system in whichthree-dimensional positional coordinates are assigned to positions inspace. For example, the time-synchronized video capture devices 104 mayhave coordinates in space and/or relative to the physical area(s) ofinterest. Alternatively or in addition, the time-synchronized videocapture devices 104 may have their orientations defined by an axisorthogonal to a reference point/plane (e.g., a face, a lens, etc.) on orassociated with the time-synchronized video capture devices 104.Overlapping and/or orthogonal orientations of the time-synchronizedvideo capture devices 104 may, as described further herein, capturevarious perspectives of the physical area(s) of interest 106.

In the example of FIG. 1A, the time-synchronized video capturemanagement system 108 is intended to represent a device that uses one ormore automated agents to control the time-synchronized video capturedevices 104 and to manage provisioning of stitched video data structuresfor playback. Generally, the time-synchronized video capture managementsystem 108 manages a plurality of video feeds from the correspondingplurality of time-synchronized video capture devices 104. The example ofFIG. 1A is intended to illustrate the time-synchronized video capturedevices 104 transferring video content to the time-synchronized videocapture management system 108 over the computer-readable medium 102. Forexample, the time-synchronized video capture management system 108 mayinclude engines and/or datastores configured to instruct thetime-synchronized video capture devices 104 to synchronize with oneanother, to identify viewer perspectives, to identify orientations ofthe fields of view 116 that relate to those viewer perspectives, tocapture video content of the physical area(s) of interest 106, to selectfields of view 116, to configure the time-synchronized video capturedevices 104, and to gather from the time-synchronized video capturedevices 104 video content of the physical area(s) of interest 106. Itshould be understood some or all of the functionality of thetime-synchronized video capture management system 108 can be sharedacross one or more of the time-synchronized video capture devices 104and/or the playback device(s) 110.

In a specific implementation, the time-synchronized video capturedevices 104 stream the video content to the time-synchronized videocapture management system 108 as the video content is captured (e.g., inreal-time), with a time-delay corresponding to pinned sound segments(e.g., n-second delayed near real-time), before, after, or aroundtime-synchronization triggers (e.g., limited batches), in batches ofpredetermined or configurable length that may be related to video ormultimedia buffer size (e.g., periodic batches), or as a single batchwhen recording is complete. In various implementations, thetime-synchronized video capture devices 104 implement a batch uploadingprocess in which saved video content is uploaded to thetime-synchronized video capture management system 108 over thecomputer-readable medium 102 at a specified time or upon occurrence of aspecified sharing trigger. In some implementations, thetime-synchronized video capture devices 104 only transfer portions ofvideo content marked or otherwise designated as relevant to a specifiedactivity. The time-synchronized video capture management system 108performs some preprocessing (such as stitching video feeds) and providesthe resulting data structure (such as a stitched video data structure)to the playback device(s) 110. The time-synchronized video capturemanagement system 108 may be implemented in a distributed fashion withfunctionality implemented on one or more of the time-synchronized videocapture devices 104 and/or the playback device(s) 110.

In a specific implementation, the time-synchronized video capturemanagement system 108 includes a master clock to which thetime-synchronized video capture devices 104 are synched. As wasmentioned above, functionality of the time-synchronized video capturemanagement system 108 may or may not be distributed across otherdevices. For example, the time-synchronized video capture devices 104could include a master time-synchronized video capture device and slavetime-synchronized video capture device(s). Continuing this example,after capturing video content of the physical area(s) of interest 106,the slave time-synchronized video capture device(s) sends video contentto the time-synchronized video capture management system 108 or to themaster time-synchronized video capture device, which provides the videocontent to the time-synchronized video capture management system 108(which may or may not be implemented on the master time-synchronizedvideo capture device) and to the playback device(s) 110.

In a specific implementation, the time-synchronized video capturemanagement system 108 marks orientations of the time-synchronized videocapture devices 104 with orientation markers. An “orientation marker,”as used herein, refers to a data structure that marks an orientation ofone of the time-synchronized video capture devices 104. The orientationmarkers may include information related to the location (globallocation, relative location relative to the physical area(s) of interest106, etc.) of a time-synchronized video capture devices 104. In variousimplementations, the orientation markers include Cartesian coordinatesand/or parameters of an axis orthogonal to a reference point/plane(e.g., a face, a lens, etc.) of a time-synchronized video capturedevices 104. Instead or in addition, the time-synchronized video capturemanagement system 108 can mark feeds associated with the visibleportions 114 of the physical area(s) of interest 106 with orientationmarkers. Advantageously, the orientation markers include data sufficientto convey camera position information to the time-synchronized videocapture management system 108 so as to enable the time-synchronizedvideo capture management system 108 to stitch the various video feedsfrom the time-synchronized video capture devices 104 together such thatthe various video feeds are correlated with the camera positions of thetime-synchronized video capture devices 104 that captured the variousvideo feeds within the stitched video data structure or a representationthereof.

In a specific implementation, the time-synchronized video capturemanagement system 108 creates a three-dimensional dome representation ofthe one or more physical area(s) of interest 106 using video contentobtained from the time-synchronized video capture devices 104 andorientation markers of orientations of the time-synchronized videocapture devices 104 (and/or the feeds associated therewith). A“three-dimensional dome representation,” as used herein, refers to arepresentation of a frame onto which the cumulative video contentcaptured by multiple time-synchronized video capture devices 104 of thephysical area(s) of interest 106 over time-synchronized periods of timeand stitched together in a three-dimensional fabric can be conceptuallyaffixed. It should be noted a “dome” need not be a smooth arc, and, asused herein, can comprise multiple flat faces. A three-dimensional domerepresentation may use one or more orientation markers to identifyorientations of video content relative to other video content includedin the data structure, or relative to some other baseline metric. Asused in this paper, the three-dimensional dome representation is assumedto be a portion of a dome, such as a half-dome, but could include a fulldome or even a sphere. More generally, the frame can be referred to as athree-dimensional (partial or full) enclosure that could correspond toany shape that has two or more inward-facing angles that correspond tocamera positions and could include any shape up to an including one thatfully encompasses a space, such as the aforementioned sphere or someother shape, such as an oval, a tube, etc.

In a specific implementation, the time-synchronized video capturemanagement system 108 creates a stitched video data structure ofphysical area(s) of interest 106 by constructively placing the variousvideo feeds on the three-dimensional enclosure in accordance withorientations of cameras of the time-synchronized video capture devices104. A stitched video data structure, as used herein, includes videocontent associated with visible portions 114 of physical area(s) ofinterest 106 arranged such that the various video feeds are oriented tomatch the orientations of the cameras used to capture the video feeds.

In a specific implementation, a stitched video representation of astitched video data structure includes one or more perspective UIelements that mark perspectives associated with the fields of view 116.In some implementations, the perspective UI elements comprise floatingvirtual objects (e.g., floating polygons, floating shapes, floatingcharacters, etc.) superimposed on portions of the stitched videorepresentation that correspond to applicable perspectives. Dependingupon implementation- and configuration-specific considerations, theperspective UI elements can be active or passive. For example, activeperspective UI elements can allow a reviewer to select perspectives inthe stitched video representation at a specific time, such as byenabling a user to “grab” a perspective UI element and rotate athree-dimensional enclosure representation to a different perspective ofthe stitched video than the one currently displayed (e.g., giving thereviewer the ability to switch between perspectives similar to those ofthe cameras that captured the perspectives). For example, passiveperspective UI elements can indicate a current perspective (e.g., bygiving a compass direction or some other indication of orientation), buthave no corresponding activation function other than to provideinformation. In a specific implementation, a mesh of active and/orpassive perspective UI elements are conceptually superimposed over thevarious video feeds in a manner that is agnostic of the video content,but they may or may not be visible, depending upon the currentperspective. For example, an active perspective UI element may only bevisible as superimposed on a video feed of a current perspective and mayor may not be visible as superimposed over an adjacent perspective.Alternatively, all or a different subset of the active perspective UIelements can be transparent so as to enable interaction with thestitched video representation, but without visibly interposingthemselves between a reviewer and the video feeds. Depending uponimplementation- and/or configuration-specific factors, as a stitchedvideo representation is moved from one perspective to the next, one ofthe video feeds can play continuously until the perspective is fullychanged to the next video. Alternatively, the currently-playing videocan be paused. Advantageously, the reviewer can move about thethree-dimensional enclosure to observe the object of interest from thevarious perspectives in an intuitive fashion that keeps the perspectiveon the object of interest without substantial jitter when changingperspective due to the time-synchronization of the video content used toform the stitched video data structure.

Depending upon implementation- and/or configuration-specificconsiderations, three-dimensional enclosure representation may or maynot accommodate different frame rates of video content from differentsynchronized video capture devices 104. As an example, athree-dimensional enclosure data structure may facilitate alignment offrames so that video content captured at 120 FPS is temporally alignedwith video content captured at 240 FPS by matching the first frame ofthe 120 FPS feed to the first and second frames of the 240 FPS feed, thesecond frame of the 120 FPS feed to the third and fourth frames of the240 FPS feed, and so forth. As another example, the number of frames inthe slower feed could be doubled or every other frame in the faster feedcould be omitted such that the stitched video representation appears tohave a consistent frame rate across each perspective or a proper subsetof the perspectives.

Sound segments can be pinned differently depending upon the apparentdistance from the focal point of a given perspective. Advantageously,zooming in or out can change the apparent distance from the focal pointand create different sound offsets and, correspondingly, changing theframes to which sound segments are pinned and, alternatively or inaddition, the volume at which the sound segments are played (e.g., thesound segments are louder when zoomed in than when zoomed out). It maybe desirable to control zoom speed or phase in sound segments to avoidzooming in so quickly that sound is played twice or zooming out soquickly that there is a span of time during which no sound segments arepinned to frames. In a specific implementation, zooming in, which canresult in previously-played sound segments being replayed, can beaccompanied by a sound similar to that heard when rewinding a cassettetape (aka a “rewind sound”). Alternatively or in addition, zooming out,which can result in skipping sound segments when the sound segments arere-pinned to later frames, can be accompanied by a static frame image asthe sound segments are played and “catch up” to the current (newlyzoomed out) frame.

In the example of FIG. 1A, the playback device(s) 110 are intended torepresent devices that use one or more automated agents and user inputsto control display of various perspectives of stitched video datastructures. The example of FIG. 1A is intended to illustrate thetime-synchronized video capture management system 108 transferringstitched video data structures, or one or more perspectives thereof, tothe playback device(s) 110. In various implementations, the playbackdevice(s) 110 comprise one or more of a mobile phone, a tablet computingdevice, and other applicable devices. In a specific implementation, thetime-synchronized video capture management system 108 streams thestitched video data structures, or one or more perspectives thereof, tothe playback device(s) 110 when or soon after the time-synchronizedvideo capture devices 104 capture video content of the physical area(s)of interest 106. Such an implementation facilitates substantiallyreal-time review, which is intended to mean real-time or real-time witha short delay, of actions in the physical area(s) of interest 106. Suchan implementation can instead or in addition facilitate concurrentreview, which is intended to mean batched provisioning of feeds thatenable the review of a current event, but with a significant delay dueto batch processing of feeds that are provided to playback device(s) 110prior to the end of the current event. Techniques described in thispaper can also improve systems that use recorded playback.

In a specific implementation, the time-synchronized video capturemanagement system 108 provides stitched live feeds, delayed live feeds,or recorded feeds to the playback device(s) 110 upon occurrence ofspecified conditions and/or in response to requests from the playbackdevice(s) 110. In any case, the time-synchronization of the videocapture devices ensures the first frames of each feed aretime-synchronized and subsequent frames have correspondinglysynchronized frames in other feeds that are stitched together.

In the diagram 100A, the playback device(s) 110 include a graphicalinterface 112 that is intended to represent an interface to facilitatereviewer interaction with stitched video representations. In variousimplementations, the graphical interface 112 comprises a graphical userinterface, including but not limited to a graphical menu-driveninterface, a touchscreen display, a voice-activated display, etc. Invarious implementations, the interactions comprise interactions with thevideo content (stop, pause, play, rewind, fast-forward, zoom, shrink,rotate, resize, etc.). The interactions may also comprise selection ofdifferent perspectives of the physical area(s) of interest 106 using theperspective UI elements.

FIG. 1B depicts a diagram 100B of an example of a time-synchronizedvideo capture environment. The diagram 100B includes thecomputer-readable medium 102, the time-synchronized video capturedevices 104, the physical areas(s) of interest 106, thetime-synchronized video capture management system 108, and the playbackdevice(s) 110. In the example of FIG. 1B, the physical area(s) ofinterest 106 is intended to show a single physical area of interest. Thephysical area(s) of interest 106 is shown as a cube, but in variousimplementations, can comprise any arbitrary or non-arbitrary size andshape.

In the example of FIG. 1B, the time-synchronized video capture devices104 comprise four applicable devices mounted on stands around thephysical area(s) of interest. As shown in FIG. 1B, a first of thetime-synchronized video capture devices 104 is oriented toward a leftface of the physical area(s) of interest 106. A second of thesynchronized video capture devices 104 is oriented toward a left-frontcorner of the physical area(s) of interest 106. A third of thesynchronized video capture devices 104 is oriented toward a front faceof the physical area(s) of interest 106. A fourth of the synchronizedvideo capture devices 104 is oriented toward a right face of thephysical area(s) of interest 106. A fifth of the time-synchronized videocapture devices 104 is mounted on a pole that extends over the top ofthe physical area(s) of interest 106 and oriented toward a top face ofthe physical area(s) of interest 106. A sixth of the time-synchronizedvideo capture devices 104 is installed on an unmanned aerial vehicle andoriented toward the top face of the physical area(s) of interest 106.

In a specific implementation, the time-synchronized video capturemanagement system 108 gathers video content from each of thetime-synchronized video capture devices 104. The time-synchronized videocapture management system 108 may identify and/or mark orientations ofthe time-synchronized video capture devices 104 in relation to thephysical area(s) of interest 106, as described previously with referenceto FIG. 1A. For example, the time-synchronized video capture managementsystem 108 can mark each of the time-synchronized video capture devices104 with Cartesian coordinates relative to the physical area(s) ofinterest 106 and/or identify angles of an axis that is orthogonal to areference point in the time-synchronized video capture devices 104 andthat connects the time-synchronized video capture devices 104 with thephysical area(s) of interest 106.

In a specific implementation, the time-synchronized video capturemanagement system 108 creates a three-dimensional cubic representationof the video content from the time-synchronized video capture devices104. The time-synchronized video capture management system 108 maycreate a stitched video representation of the one or more areas ofinterest using the three-dimensional cubic orrectangular-on-a-subset-of-sides representation of the physical area(s)of interest 106. Alternatively or in addition, the time-synchronizedvideo capture management system 108 can create a three-dimensionalspherical or semi-spherical (e.g., dome-shaped) representation of thevideo content from the time-synchronized video capture devices 104.Alternatively or in addition, the time-synchronized video capturemanagement system 108 can create a three-dimensional polyhedron (e.g.,an octagonal prism) or portion thereof representation of the videocontent from the time-synchronized video capture devices 104 or athree-dimensional shape that is equivalent in shape to tiling on asphere or semi-sphere. The time-synchronized video capture managementsystem 108 may provide the stitched video representation of the physicalarea(s) of interest 106 to the playback device(s) 110.

In the example of FIG. 1B, the playback device(s) 110 are intended toillustrate a way to allow a reviewer to interact with the stitched videorepresentation. In various implementations, the interactions maycomprise interactions with the video content (stop, pause, play, rewind,fast-forward, zoom, shrink, rotate, resize, etc.). The interactions mayalso comprise selection of different perspectives of the physicalarea(s) of interest 106 using the perspective UI elements.

FIG. 1C depicts a diagram 100C of an example of a time-synchronizedvideo capture environment. The diagram 100C includes thecomputer-readable medium 102, the time-synchronized video capturedevices 104, the physical areas(s) of interest 106, thetime-synchronized video capture management system 108, and the playbackdevice(s) 110.

In the example of FIG. 1C, the physical area(s) of interest 106 comprisemultiple discrete physical area of interest. The physical area(s) ofinterest 106 are shown as multiple cubes, but in variousimplementations, can comprise a plurality of arbitrary or non-arbitrarysizes and shapes. In a specific implementation, the time-synchronizedvideo capture devices 104 operate to capture different portions of thephysical area(s) of interest 106 depending on the location of an objectwithin the physical area(s) of interest. For example, a portion of thetime-synchronized video capture devices 104 can be configured to movewith an object in the physical area(s) of interest 106. To facilitatethis movement, the time-synchronized video capture devices 104 may bemounted on a platform that moves along with an object in the physicalarea(s) of interest. Depending upon implementation- and/orconfiguration-specific considerations, the time-synchronized videocapture devices 104 can be configured to change their fields of view toaccommodate an object moving in the physical area(s) of interest 106.For example, the time-synchronized video capture devices 104 can beconfigured to rotate around a base to follow an object within thephysical area(s) of interest 106. Alternatively or in addition, thetime-synchronized video capture devices 104 and/or their fields of viewcan follow an object using a device (a fob, a beacon, etc.) on theobject in the physical area(s) of interest 106.

FIG. 1D depicts a diagram 100D of an example of a time-synchronizedvideo capture environment. The diagram 100D includes thetime-synchronized video capture devices 104, the physical areas(s) ofinterest 106, and the playback device(s) 110. The diagram 100D furtherincludes an object 118 and a reviewer 120. In this example, the object118 appears to be a baseball player at bat. The physical area(s) ofinterest 106 may comprise a relevant area around the object 118 wherethe object's actions (e.g., batting a ball) is of interest. Thetime-synchronized video capture devices 104 may capture actions of theobject 118 using the techniques described herein. The reviewer 120 mayreview, on the playback device 110, a stitched video representation ofthe physical area(s) of interest 106 at a specified time (e.g., when thebatter, which is object 118 in this example, is attempting to hit aball).

Example Time-Synchronized Video Capture Device

FIG. 2 depicts a diagram 200 of an example of a time-synchronized videocapture device. The diagram 200 includes video capture engine(s) 202,video camera(s) 204, a network interface 206, a graphical interface 208,sensor(s) 210, a video content datastore 212, and a housing 214. In theexample of FIG. 2, the video capture engine(s) 202 include a videocamera control engine 222, a field of view management engine 224, a timesynchronization management engine 226, and a video content managementengine 228.

The video camera control engine 222 is intended to represent a devicethat uses one or more automated agents to process instructions to managethe video camera(s) 204. Advantageously, the automated agents act onbehalf of a larger system to ensure each video camera fulfils its roleas part of a set of time-synchronized video capture devices withouthuman interaction, which would tend to make impossible the maintenanceof time-synchronization and/or the generation of feeds that can bestitched together properly. The video camera control engine 222 mayinclude drivers and/or control circuitry to instruct the video camera(s)204 to initiate, end, etc. recording of video content. Instructions tomanage the video camera(s) 204 may come from time-synchronized videocapture management system over the network interface 206, or from rulesinput to the device manually (e.g., via a flash memory device orgraphical user interface).

The field of view management engine 224 is intended to represent adevice that uses one or more automated agents to establish a field ofview suitable for capturing a portion of a physical area of interest(see, e.g., FIGS. 1A-1D). In a specific implementation, the field ofview management engine 224 is responsive to adjustments to physicalplacement of a time-synchronized video capture device and/or orientationor configuration of the video camera(s) 204. For example, a human canplace a time-synchronized video capture device in an attempt toestablish a desirable field of view. In this example, the field of viewmanagement engine 224 provides feedback regarding whether the field ofview is ideal or adequate. In the example in which a human is adjustingthe orientation of the device, the feedback can be provided via thegraphical interface 208 (or through some other machine-to-humaninterface). In an example in which the time-synchronized video capturedevice is mounted on an automated mobile platform (not shown), the fieldof view management engine 224 can instruct a mobile platform controllerto move the mobile platform so as to achieve a desirable orientation forthe video camera(s) 204. In this example, feedback may or may not benecessary depending upon how the mobile platform controller receivesinstructions and how precise the mobile platform is in positioning thevideo camera(s) 204. The field of view management engine 224 can workacross devices to coordinate field of views for multipletime-synchronized video capture devices, either by establishing a firsttime-synchronized video capture device as the baseline (unmoving) deviceor by dynamically adjusting any or all of the various devices to capturea desired combination of fields of view.

In a specific implementation, the field of view management engine 224receives parameters about an intended field of view (e.g., relevantangles, relevant objects to focus on, relevant distances from objects,etc.). Corresponding instructions can be relayed to a human or mobileplatform for initial placement of the time-synchronized video capturedevice, as described above, and to the video camera(s) 204, via thevideo camera control engine 222, to rotate, move, etc. so that anappropriate field of view is displayed thereon. For example, assumingthe video camera(s) 204 can adjust field of view (e.g., via zoom orother controls), the video camera control engine 222 can instruct thevideo camera(s) 204 to modify field(s) of view in response toinstructions from the field of view management engine 224. The intendedfield of view may, but need not, be related to a desired perspective ofphysical area(s) of interest. Advantageously, the automated agents canameliorate the risk associated with humans providing on-the-fly field ofview adjustments, which can result in feeds that are not optimal forstitching.

The time synchronization management engine 226 is intended to representa device that uses one or more automated agents to process instructionsto synchronize video capture by the video camera(s) 204 with videocameras of other time-synchronized video capture devices, or imagescaptured therethrough. Advantageously, automated agents can accomplishtime-synchronization across multiple time-synchronized video capturedevices in a manner that humans simply cannot do. For example, formultiple 120 FPS cameras to be properly time-synchronized, humans at thevarious devices who press start at the “same time” would be off bypotentially dozens of frames.

In a specific implementation, the time synchronization management engine226 uses time synchronization triggers to begin or end recording at atime before, at, or after the time synchronization trigger is activated.With respect to beginning or ending a recording before the timesynchronization trigger is activated, a feed can be buffered to enable arecording to start or end at a time prior to the activation. Indeed, theact of “starting a recording” can be problematic without relativelyinstantaneous activation of a camera, making buffering prior to startingthe recording particularly desirable in some implementations. Forexample, feed(s) from the video camera(s) can be buffered or otherwisestored in the video content datastore 212 at a specified time and/orbefore, upon, or after the occurrence of a specified timesynchronization trigger. Instead or in addition, the video camera(s) 204can record continuously, but only buffer a subset of the recordedcontent (or store a subset of the recorded content for batchprocessing). In various implementations, the time synchronizationmanagement engine 226 monitors (actively or passively) the videocamera(s) 204, network interface 206, the graphical interface 208,and/or the sensor(s) 210 for events associated with atime-synchronization trigger.

A “time synchronization trigger,” as used herein, refers to an event,the occurrence of which results in the video camera(s) 204 initiatingvideo recording or enabling the time synchronization management engine226 to identify a starting frame for a feed stored in the video contentdatastore 212 that was previously or concurrently captured by the videocamera(s) 204. The occurrence of the event can also be referred to as atime synchronization stimulus. Examples of time synchronization stimuliinclude clock signals (e.g., from timers/alarms or clocks), specificsounds (e.g., the sound of a bat or golf club hitting a ball, the soundof a bat or golf club swinging, the sound of a punch/kick, the sound ofice skates moving across an ice rink, etc.), electromagnetic signals(e.g., visible light enabling identification of specific actions in anactivity or responses to an event using machine vision, such asdetermining a person was punched by observing a bodily reaction to thepunch), motion (e.g., detected using a motion-detector). Timesynchronization stimuli can also come from a device (e.g., a fob, abeacon, etc.) that is associated with a physical position of a subject.Time synchronization stimuli may or may not be detected by the sensor(s)210. A relatively simple time synchronization trigger is simply anexplicit start time for an event. Advantageously, the timesynchronization management engine 226 can use the time synchronizationtriggers to mark or otherwise designate portions of video contentrelevant to an activity. The video camera control engine 222 instructsthe video camera(s) 204 to initiate recording in response to a timesynchronization trigger, which is at least an explicit instruction tostart recording, specified by the time synchronization management engine226.

The video content management engine 228 is intended to represent adevice that uses one or more automated agents to process instructions tomanage video content captured by the video camera(s) 204.Advantageously, the video content management engine 228 can manage videocontent in near real-time (or at least has the capability of managingvideo content in near real-time). For example, in some implementations,video content can be recorded continuously both before a timesynchronization trigger and after, but the video content managementengine 228 may only transfer portions of video content that residewithin a rules-based time span associated with a time synchronizationtrigger, before or after a synchronization trigger has occurred. Thevideo content management engine 228 can control the contents of thevideo content datastore 212, including creating reading, updating, ordeleting data structures.

Depending upon implementation- and/or configuration-specificconsiderations, time synchronization trigger can indicate all videocontent captured by the video camera(s) 204 is to be streamed or sent inbatches, or a proper subset of the video content is to be streamed orsent in batches. The video content management engine 228 may store videocontent captured by the video camera(s) 204 in the video contentdatastore 212. Depending upon implementation- and configuration-specificconsiderations, the video content management engine 228 stream, with orwithout delay, or otherwise transmit, such as in one or more batchfiles, over the network interface 206 video content captured by thevideo camera(s) 204. Advantageously, as described, the video contentmanagement engine 228 can stream or send in batches all or a subset ofvideo content buffered or otherwise stored in the video contentdatastore 212 with relatively little preprocessing. Depending uponimplementation- and/or configuration-specific considerations, the videocontent management engine 228 can allow a human or automated agent toedit (filter, etc.) video content after the video content has beencaptured by the video camera(s) 204 via the graphical interface 208.However, because automated agents prevent frame misalignment that isinherent if humans have control over managing video content and speedthe management operations sufficiently to enable real-time or nearreal-time stitching of the various feeds, in at least someimplementations, the automated agents are responsible for framealignment.

In a specific implementation, the video camera(s) 204 have a staticframe rate that is the same for each of the video camera(s) 204. In thisimplementation, other video cameras capturing visible portions of aphysical area of interest (see, e.g., FIGS. 1A-1D) can have the samestatic frame rate. The advantage of static frame rates acrosstime-synchronized video capture devices is that the each frame of eachcamera is synchronized in time with each corresponding frame of theother cameras of interest. That is, frame 3 from the video feed of afirst camera is synchronized in time with frame 3 from the video feed ofa second camera (and, in fact, of every other camera of interest).Advantageously, frame-by-frame synchronization substantially reducespre-processing requirements for the various feeds, and makes it easy tochoose a subset of the various feeds that is of interest and can bereadily aligned. When the first frame is known at the time synchronizedvideo capture device, the video content management engine 228 canconceptually clip all frames before the first frame and starttransmitting the feed (via streaming or in batches) starting with thefirst frame. Similarly, when the last frame is known at the timesynchronized video capture device, the video content management engine228 can conceptually clip all frames after the last frame and ceasetransmission of the feed when the last frame is reached.

In an alternative implementation, the video camera(s) 204 includecameras with different (configurable or static) frame rates. In thisimplementation, other video cameras capturing visible portions of aphysical area of interest (see, e.g., FIGS. 1A-1D) can also havedifferent (configurable or static) frame rates. As an example, a firstof the video camera(s) 204 may capture images at 120 frames per second(FPS), while a second of the video camera(s) 204 (or a camera of adifferent time-synchronized video capture device) may capture images at240 FPS. The time synchronization correlates the first frame of each ofthe various feeds with the understanding some frames may be dropped(from higher FPS feeds) or added (to lower FPS feeds). Sub-optimalstitching can occur when the feeds have mismatched frames, such as ifone feed is 60 FPS and another feed is 100 FPS. In this example, thesecond frame of the 60 FPS feed is temporally located between the secondand third frames of the 100 FPS feed. Depending upon implementation- andconfiguration-specific considerations, it may be desirable to find alowest common denominator (LCD), which in the example of 60 FPS and 100FPS is 10, and drop any frames that do not align with the 10 FPSframework. For example, if a coach is analyzing the swing of a golfer,the coach may find it more useful to sacrifice FPS to ensure the framesare, for practical purposes, perfectly time synchronized. If applicable,sound segments are at least conceptually pinned to the LCD frames. Inother instances, it may be desirable to keep the frame rate higher toallow the feed to flow more like a modern movie, with the risk ofjittery stitches or more blurry frames. For example, if a reviewerdoesn't want to watch a frame with a highest captured frame rate, thefeeds can be played at the speed of the current field of view such thatwhen viewing a physical area of interest with a first view captured at60 FPS, the feed is played at 60 FPS, but when changing to a second viewassociated with a 100 FPS camera, the feed is played at 100 FPS.(Changing between fields of view is discussed later in association withthe playback device.) In other instances, a static FPS playback can beselected such that all feeds are reviewed at, e.g., 60 FPS, even if oneor more are captured at 100 FPS. The video content management engine 228may or may not be responsible for providing feeds at a prescribed FPS,depending upon implementation- and/or configuration-specificconsiderations; the adjustments can also be made at a time-synchronizedvideo capture management system (described later).

In the example of FIG. 2, the video camera(s) 204 are intended torepresent devices configured to capture video of portions of physicalarea(s) of interest. In some implementations, the video camera(s) 204include lenses, focal hardware, sensors, storage media, etc. forgathering and/or storing video content. The video camera(s) 204 mayinclude specialized and/or dedicated video cameras characterized bymultiple focal points, higher shutter speeds than mobile phone cameras,etc. The video camera(s) 204 may include a depth camera (e.g., acombination of a depth sensor and a camera) configured to capturethree-dimensional video content. As an example, the video camera(s) 204may include a depth sensor that senses contours at specified distancesaway from the video camera(s) 204. The video camera(s) 204 may use datafrom the depth sensor in conjunction with image data from opticalhardware to identify three-dimensional attributes of physical area(s) ofinterest. Alternatively, the video camera(s) 204 may includestereoscopic cameras, generally implemented as two or more light sensorswith sufficient spatial diversity to capture a stereoscopic image.

In the example of FIG. 2, the network interface 206 is intended torepresent drivers and/or control circuitry that facilitatescommunication over a computer-readable medium. The network interface 206may comprise a device or other port of the time-synchronized videocapture device 200.

In the example of FIG. 2, the graphical interface 208 is intended torepresent drivers and/or control circuitry for providing output toand/or receiving input from a human. The graphical interface 208 isoptional because the time-synchronized video capture device can be fullyautomated in some implementations. A specific example of a graphicalinterface 208 is a smartphone screen by which a person enters aninstruction to allow the smartphone to act as a time-synchronized videocapture device.

In the example of FIG. 2, the sensor(s) 210 are intended to representhardware, drivers, and/or control circuitry configured to sense aphysical property around the time-synchronized video capture device 200.In various implementations, the sensor(s) 210 include audio sensorsconfigured to sense sounds. The audio sensors may be configured toidentify specific sounds associated with various activities (e.g., thesound of a bat or golf club hitting a ball, the sound of a bat or golfclub swinging, the sound of a punch/kick, the sound of ice skates movingacross an ice rink, etc.). The sensor(s) 210 may include machine visionsensors that recognize physical attributes of the environment around thetime-synchronized video capture device 200. The sensor(s) 210 mayinclude motion detection sensors that detect specified motions. In someimplementations, the sensor(s) 210 include wireless network sensors(Bluetooth sensors, Bluetooth Low Energy sensors, Radio FrequencyIdentification (RFID) sensors, etc.) that identify the physicalpositions of an object of a video recording. The sensor(s) 210 mayprovide sensor data that, as discussed previously, may be used as thebasis of time-synchronization triggers. To the extent captured sensordata is incorporated into a multimedia feed, the video capture engine(s)202 can be characterized as “multimedia capture engine(s),” with acorresponding sensor control engine and media content management enginefor each sensor or group of sensors.

In the example of FIG. 2, the video content datastore 212 is intended torepresent a datastore configured to store video content captured by thevideo camera(s) 204. The video content datastore 212 includes a videofeed. Depending upon implementation- and/or configuration-specificconsiderations, the video feed can be augmented with an audio or othertype of feed and characterized as a “multimedia feed.” The sensor(s) 210can also have corresponding media storage datastores (not shown) if thesensor(s) 210 have corresponding discrete feeds.

In the example of FIG. 2, the housing 214 is intended to represent aprotective housing for the various components of a time-synchronizedvideo capture device. Depending upon implementation- and/orconfiguration-specific considerations, the housing 214 can be asmartphone housing, a laptop computer housing, or some other devicehousing for the various components of a time-synchronized video capturedevice. In a specific implementation, the housing 214 includes an actionhousing, which is intended to refer to a housing used to protect acamera and/or sensors used in conjunction with a camera, and whichfacilitates mounting the time-synchronized video capture device inassociation with a particular activity. An action housing can also beimplemented as part of a platform (such as a cavity in which a timesynchronized video capture device is received or a mount for a timesynchronized video capture device). The action housing can varydepending upon the environment and/or depending upon implementation-and/or configuration-specific considerations.

In an example of operation, a human positions a time synchronized videocapture device to face a physical area of interest. (Automated mobileplatforms may eliminate the need for a human to position the device.) Itmay be noted the device may or may not be time-synchronized at the timeof deployment, but it will be, which is why it is referred to as timesynchronized throughout this example. The field of view managementengine 224 provides instructions via the network interface 206 and/orthe graphical user interface 208 to adjust the orientation of the timesynchronized video capture device and/or provides instructions to thevideo camera control engine 222 to capture a relevant field of view of aphysical area of interest without further human interaction. The videocamera control engine 222 receives over the network interface 206 and/orthe graphical interface 208 instructions to activate (wake up, beginrecording, or the like) the video camera(s) 204; images captured by thevideo camera(s) 204 are stored in the video content datastore 212. Thetime synchronization management engine 226, which receives a timesynchronization trigger via the network interface 206 and/or thegraphical interface 208, detects a time synchronization stimulusassociated with the time synchronization trigger from a clock (notshown), the video camera(s) 204, and/or the sensor(s) 210, and providesinstructions to the video camera(s) 204 to begin recording and/orinforms the video content management engine 228 the stimulus wasdetected to enable the identification of a start frame for a feed, whichmay or may not involve preprocessing a feed. The video contentmanagement engine 228 instructs the network interface 206 to stream thevideo content with or without a delay or to transmit the feed in batchesfrom the video content datastore 212.

Example Playback Device

FIG. 3 depicts a diagram 300 of an example of a playback device. Thediagram 300 includes playback engine(s) 302, a network interface 304, agraphical interface 306, a stitched video datastore 310, and a housing318. In the example of FIG. 3, the playback engine(s) 302 include astitched video representation processing engine 312, a stitched videorepresentation perspective engine 314, and a user interaction managementengine 316.

In various implementations, the playback device(s) 110 support aprocess, an application, a webpage, or the like implemented thereon thatfacilitates interaction with stitched video representations of thephysical area(s) of interest 106.

The stitched video representation processing engine 312 is intended torepresent a device that uses one or more automated agents to processstitched video representations of one or more physical areas ofinterest. In a specific implementation, the stitched videorepresentation processing engine 312 can received over the networkinterface 304 and display on the graphical interface 306 a stitchedvideo (with or without perspective UI elements that allow a reviewer toselect a perspective or a subset of perspectives of a physical area ofinterest corresponding to fields of view of cameras that captured theapplicable video content). Alternatively, the stitched videorepresentation processing engine 312 can receive stitched video via aninterface (not shown) other than the network interface 304, such as aUSB port, and/or display the stitched video via an interface (not shown)other than or in addition to the graphical user interface 306, such asvia a web page.

In a specific implementation, the stitched video representationprocessing engine 312 provides instructions to the graphical interface306 to display a stitched video representation of one or more physicalareas of interest. For example, the stitched video representationprocessing engine 312 can communicate with an application displayed onthe graphical interface 306 to display a stitched video representationin the application. The stitched video representation processing engine312 may access one or more Application Programming Interfaces (APIs)supported by the application in order to provide specific images,renderings, etc. to display the stitched video representation on thegraphical interface 306.

The stitched video representation perspective engine 314 is intended torepresent a device that uses one or more automated agents to facilitatemanagement of a perspective of a stitched video representation displayedon the graphical interface 306. Advantageously, the one or moreautomated agents can interpret input from a human or agent of the humanto display a perspective of the stitched video representation inaccordance with the input. For example, the automated agents of thestitched video representation perspective engine 314 can interpretinteractions with the graphical interface 306 to provide a desiredperspective stitched video representation. To facilitate humaninteraction, the stitched video representation can include perspectiveUI elements, such as a graphical “handle” with which to grab a stitchedvideo and spin the stitched video from a first perspective to a secondperspective.

In a specific implementation, the automated agents process playbackinstructions, such as instructions to stop playback, to pause playback,to initiate or continue playback, to rewind playback, and tofast-forward playback. Alternatively or in addition, the automatedagents can process instructions to modify a size or an area of astitched video representation. As examples, the automated agents of thestitched video representation perspective engine 314 may processinstructions to zoom, shrink, rotate, resize, or otherwise modify astitched video representation. Depending upon implementation- and/orconfiguration-specific considerations, changing perspective can resultin a corresponding change to sound offsets or volume or other aspects ofa multimedia representation.

The user interaction management engine 316 is intended to represent adevice that uses one or more automated agents to receive userinteractions from the graphical interface 306 and provides correspondinguser interaction instructions to other engines. In a specificimplementation, the user interaction management engine 316 implementsinstructions to control time-synchronized video capture device(s). Forexample, the user interaction management engine 316 may processinstructions to control frames per second (FPS), shutter speeds,active/inactive states, light meters, heights, zooms, rotations, to nameseveral, of time-synchronized video capture device(s). In someimplementations, the user interaction management engine 316 allows areviewer to zoom in and/or focus on a subject in a stitched videorepresentation.

In the example of FIG. 3, the network interface 304 is intended torepresent drivers and/or control circuitry for providing data to andsending data from a computer-readable medium. The network interface 304may comprise a device or port of a playback device. Alternatively or inaddition, a playback device can include other interfaces, such as USBports, multiple network interface types (e.g., WiFi, 4G, Ethernet,etc.), and DVD players, to name a few.

In the example of FIG. 3, the graphical interface 306 is intended torepresent drivers and/or control circuitry for receiving user input fromand/or displaying content for a reviewer. Alternatively or in addition,a playback device can include other input devices, such as keyboards,mice, and gesture sensors, to name a few, and other display devices,such as peripherals, speakers, and vibrators, to name a few.

In the example of FIG. 3, the stitched video datastore 310 is intendedto represent a datastore that stores stitched video data structuresassociated with one or more physical areas of interest. Depending uponimplementation- or configuration-specific factors, the stitched videodatastore 310 can rebuffer a stitched video representation whenperspective changes, using a stitched video data structure to constructstitched video representations as a function of the present perspective.Alternatively, the stitched video datastore 310 can buffer all possibleperspectives, which correspond to the number of feeds used to constructthe stitched feed, and switch to the appropriate buffer when a desiredperspective changes is implicated via inputs from a reviewer or agent ofthe reviewer.

In the example of FIG. 3, the housing 318 is intended to represent aprotective housing for the various components of a time-synchronizedvideo playback device. Depending upon implementation- and/orconfiguration-specific considerations, the housing 318 can be asmartphone housing, a laptop computer housing, or some other devicehousing for the various components of a time-synchronized video playbackdevice.

In an example of operation, the network interface 304 receives one ormore stitched video representation data structures. The stitched videorepresentation processing engine 312 stores a stitched videorepresentation in the stitched video datastore 310. The stitched videorepresentation processing engine 312 gathers, based on user instructionsreceived via the graphical interface 306 and processed by the userinteraction management engine 316, user instructions to interact withstitched video representations. Commands that correspond to the userinteractions are be provided to the stitched video representationprocessing engine 312, which may instruct the graphical interface 306 todisplay or modify the display of the stitched video representation. Theplayback engine(s) 302 operate to display a stitched videorepresentation of one or more physical areas of interest in accordancewith the relevant stitched video representation data structure and therelevant instructions via the graphical interface 306.

Example Time-Synchronized Video Capture Management System

FIG. 4 depicts a diagram 400 of an example of a time-synchronized videocapture management system. The diagram 400 includes video capturemanagement engine(s) 402, a time-synchronization trigger datastore 404,an orientation marker datastore 406, a perspective UI element datastore408, and a video content datastore 410. In the example of FIG. 4, thevideo capture management engine(s) 402 include a time synchronizationmanagement engine 412, a time-synchronized video capture devicemanagement engine 414, a playback device management engine 416, an areaidentification engine 418, a three-dimensional dome representationintegration engine 420, and a stitched video representation managementengine 422. A time-synchronized video capture management system caninclude other engines and datastores (not shown), such as a metadatadatastore that stores metadata of time-synchronized video capturedevices (e.g., model name, model number, etc.).

The time synchronization management engine 412 is intended to representa device that uses one or more automated agents to managetime-synchronization triggers as the basis of video capture of aphysical area of interest by time-synchronized video capture devices.For example, the time synchronization management engine 412 may manageone or more specified times and/or one or more specified physicalconditions that relate to video capture of a given event/physical areaof interest.

In a specific implementation, the time synchronization management engine412 also uses one or more automated agents to manage one or morespecified sounds relevant to an activity that is subject to videocapture. Examples of sounds that may be managed include the sound of abat or golf club hitting a ball, the sound of a bat or golf clubswinging, the sound of a punch/kick, the sound of ice skates movingacross an ice rink, etc. In various implementations, the timesynchronization management engine 412 manages machine vision techniques,such as techniques used to identify specific actions in an activity,that form the basis of time-synchronization triggers. The timesynchronization management engine 412 may manage arrangements ofspecific signals from a device (e.g., a fob, a beacon, etc.) that isassociated with a physical position of a subject of a video recordingand that is used as the basis of one or more time-synchronizationtriggers.

The time-synchronized video capture device management engine 414 isintended to represent a device that uses one or more automated agents tomanage time-synchronized video capture devices. For example, thetime-synchronized video capture device management engine 414 can providetime-synchronized video capture devices with time-synchronizationtriggers. The time-synchronized video capture device management engine414 may further provide time-synchronized video capture devices withinstructions to control video camera(s) and/or sensors toward a physicalarea of interest. As examples, the time-synchronized video capturedevice management engine 414 may provide time-synchronized video capturedevices with specific zoom settings, specific orientations towardphysical objects, etc. The time-synchronized video capture devicemanagement engine 414 may provide time-synchronized video capturedevices with time-synchronization triggers and instructions to respondto the time-synchronization triggers. The time-synchronized videocapture device management engine 414 may provide instructions to controltime-synchronized video capture device(s). For example, the playbackdevice management engine 416 may provide the time-synchronized videocapture devices with instructions to capture a specific scene, aspecific physical area of interest, or a specific activity.

The playback device management engine 416 is intended to represent adevice that uses one or more automated agents to manage playbackdevice(s). For example, the playback device management engine 416 mayprovide playback device(s) with instructions to set up a specific scene,a specific physical area of interest, or a specific activity. In aspecific implementation, the playback device management engine 416provides playback device(s) with stitched video representations of aphysical area of interest. In various implementations, the playbackdevice management engine 416 manages reviewer accounts of reviewers whouse playback device(s). The playback device management engine 416 mayreceive instructions to control time-synchronized video capturedevice(s). As examples, playback device management engine 416 mayreceive instructions to control frames per second (FPS), shutter speeds,active/inactive states, light meters, heights, zooms, rotations, etc. oftime-synchronized video capture device(s).

The area identification engine 418 is intended to represent a devicethat uses one or more automated agents to identify one or more physicalareas of interest subject to video capture. The time-synchronized videocapture device management engine 414 may receive from the areaidentification engine 418 information about a specific area of interestthat is to be subject to video capture. The area identification engine418 may use geolocational data, information from meshes/contours/etc.and/or other information to identify parameters of a physical area ofinterest.

The three-dimensional dome representation integration engine 420 isintended to represent a device that uses one or more automated agents tointegrate video content of physical area(s) of interest taken fromtime-synchronized video capture devices into a three-dimensional domerepresentation of the physical area(s) of interest. Thethree-dimensional dome representation integration engine 420 may gathervideo content from the video content datastore 410. In someimplementations, the three-dimensional dome representation integrationengine 420 identifies one or more orientations of video content relativeto the physical area(s) of interest. The three-dimensional domerepresentation integration engine 420 may mark specific video contentwith orientation markers obtained from the orientation marker datastore406. The three-dimensional dome representation integration engine 420may arrange video content according their orientations relative tophysical area(s) of interest. As an example, the three-dimensional domerepresentation integration engine 420 may project video content onto amap or a projection that arranges the video content therein according tothe orientations of the time-synchronized video capture devices thatcaptured that video content.

Depending upon implementation- or configuration-specific factors, thethree-dimensional dome representation integration engine 420 may beconfigured to accommodate different frame rates of video content fromdifferent synchronized video capture device(s). As an example, thethree-dimensional dome representation integration engine 420 may alignframes so that video content captured at 120 FPS is temporally alignedwith video content captured at 240 FPS. In some implementations, thethree-dimensional dome representation integration engine 420 may matchtimestamps on frames to ensure that they are aligned in accordance withthe video capture. Instead or in addition, the three-dimensional domerepresentation integration engine 420 may align portions of frames otherthan timestamps.

The stitched video representation management engine 422 is intended torepresent a device that uses one or more automated agents to combinevideo content into a stitched video representation of physical area(s)of interest. For example, the stitched video representation managementengine 422 may use orientation markers as the basis of perspective UIelements on each item of video content such that different perspectivesare stitched together into a continuous fabric. In variousimplementations, the stitched video representation management engine 422gathers perspective UI elements (boxes, shapes, virtual objects, otherUI elements, etc.) from the perspective UI element datastore 408. Inconjunction with the three-dimensional dome representation integrationengine 420, the stitched video representation management engine 422 cansuperimpose the perspective UI elements over portions of thethree-dimensional dome representation that correspond to orientation(s)of specific video content. The stitched video representation may includethe perspective UI elements as portions that intuitively enable areviewer to change orientation of a video capture of physical area(s) ofinterest by interacting with the perspective UI elements.

In the example of FIG. 4, the time-synchronization trigger datastore 404is intended to represent a datastore of time-synchronization triggers,the orientation marker datastore 406 is intended to represent adatastore of orientation markers, the perspective UI element datastore408 is intended to represent a datastore of perspective UI elements, andthe video content datastore 410 is intended to represent a datastore ofvideo content. Omitted from the discussion of FIG. 4 is embodiments thatincorporate sound or other components into a video, which was describedpreviously and is applicable here as described.

In an example of operation, the time-synchronized video capturemanagement system creates a stitched video representation of physicalarea(s) of interest based on video content taken from time-synchronizedvideo capture devices arranged around the physical area(s) of interest.The time synchronization management engine 412 identifies and/or gathersrelevant time-synchronization triggers for a specific activity/physicalarea(s) of interest from the time-synchronization trigger datastore 404.The time synchronization management engine 412 may provide thetime-synchronization triggers to time-synchronized video capturedevices. The time-synchronized video capture devices may have been setup manually in coordination with automated agents the time-synchronizedvideo capture device management engine 414.

Alternatively, where the time-synchronized video capture devices aremounted on a mobile platform, the time-synchronized video capture devicemanagement engine 414 can provide instructions to the time-synchronizedvideo capture devices to move to locations appropriate for a givenphysical area of interest, and to adjust orientations and settings asappropriate, without manual placement. Regardless of whether placementwas manual, the time-synchronized video capture device management engine414 provides the time-synchronized video capture devices with specificconfigurations/settings/etc. to capture physical area(s) of interest, ifnecessary to properly capture the physical area(s) of interest. Notably,manual placement with no automated feedback is generally insufficient toproperly capture a physical area of interest, though it may be possibleto use certain tools (such as levels and range finders) to enable humansto perform the placement task without automated feedback from thetime-synchronized video capture device management engine 414. The areaidentification engine 418 may assist in configuring thetime-synchronized video capture devices for a specified physical area ofinterest. The time-synchronized video capture devices provide videocontent of the area(s) of interest; and the time-synchronized videocapture device management engine 414 stores the video content in thevideo content datastore 410. Each item of time-synchronized videocontent may correspond to a field of view (i.e., the field of view of arelevant time-synchronized video capture device) of the physical area(s)of interest. Each field of view may provide a different perspective ofthe physical area(s) of interest.

In this example of operation, the three-dimensional dome representationintegration engine 420 identifies orientations for each field of view.The orientations may be associated with a viewer perspective of eachfield of view. The three-dimensional dome representation integrationengine 420 may further gather orientation markers from the orientationmarker datastore 406 and may mark each orientation with an orientationmarker. In various implementations, the three-dimensional domerepresentation integration engine 420 integrates the time-synchronizedvideo content and the orientation markers into a three-dimensional domerepresentation of the one or more areas of interest. As noted herein,the three-dimensional dome representation may be configured to arrangethe time-synchronized video content in accordance with the orientationmarkers.

In this example of operation, the stitched video representationmanagement engine 422 creates a stitched video representation of the oneor more areas of interest using the three-dimensional domerepresentation. The stitched video representation may be configured tofacilitate display of any of the time-synchronized video content at aspecific time. As an example, the stitched video representation may beconfigured to allow a reviewer to view any perspective of the physicalarea(s) of interest captured by any of the time-synchronized videocapture devices at a given time. The stitched video representation, forinstance, may allow a reviewer to view a top view of physical area(s) ofinterest at a given time, and then change to a side view of the physicalarea(s) of interest at the same time. The stitched video representationmanagement engine 422 may incorporate perspective UI elements to markvarious orientations of video content that form the basis of thestitched video representation. For instance, the stitched videorepresentation management engine 422 may incorporate virtual objects(boxes, shapes, etc.) to mark orientations and/or allow reviewers tochange perspectives in relation to physical area(s) of interest. Theplayback device management engine 416 provides the stitched videorepresentation to one or more playback devices for display by thoseplayback devices.

Flowcharts of Example Methods of Operation

FIG. 5 shows a flowchart 500 of an example of a method for capturingtime-synchronized video content of a visible portion of physical area(s)of interest. It is noted the flowchart 500 may include a greater or alesser number of operations than those explicitly depicted, and that notall operations in the flowchart may be necessary for a specificimplementation. The method shown in the flowchart 500 may be carried outby a time-synchronized video capture device.

In the example of FIG. 5, the flowchart 500 starts at module 502 withreceiving video capture parameters associated with a physical area ofinterest. In some implementations, the video capture parameters maycomprise FPS, shutter speeds, whether a device is to be in an active orinactive state, whether light meters are to be activated, and otherparameters such as heights, zooms, rotations, etc. of thetime-synchronized video capture device(s). The purpose of the parametersis to enable a time-synchronized video capture device to configureitself to capture a visible portion of an area of interest in a desiredmanner. The visible portion may correspond to a field of view of thetime-synchronized video capture device. Depending upon implementation-or configuration-specific considerations, the physical area of interestmay comprise a single area of interest or a plurality of continuous ordiscrete areas of interest.

In the example of FIG. 5, the flowchart 500 continues at module 504 withreceiving at least one time-synchronization trigger. Thetime-synchronization trigger may comprise an event, the occurrence ofwhich results in a time-synchronized video capture device initiatingvideo content recording, identifying a starting frame of previouslycaptured video content, initiating buffering of video content, orstarting a count-down to when the time-synchronized video capture devicewill begin buffering video content. In some implementations, thetime-synchronization trigger comprises specified times and/or specifiedphysical conditions relating to video capture of a given event.

In the example of FIG. 5, the flowchart 500 continues at module 506 withdetecting occurrence of a stimulus related to the time-synchronizationtrigger. For example, the time synchronization trigger may be based onsound, vision, motion, signals from a device (e.g. a fob or beacon), toname several.

In the example of FIG. 5, the flowchart 500 continues at module 508 withgathering video content of a visible portion of the physical area ofinterest in response to detecting the stimulus. For example, thetime-synchronized video capture device may initiate recording inresponse to the occurrence of the condition, identify a first frame ofpreviously captured video content, or the like. The video content of thevisible portion of the physical area of interest obtained by thetime-synchronized video capture device may be time-synchronized in thatthe starting frame of the relevant video content feed is the same as thestarting frame of the relevant video content of anothertime-synchronized video capture device, but with a differentperspective. Advantageously, the use of the time-synchronized videocontent may allow a stitched video representation of the physicalarea(s) of interest to be obtained without the use of complex processingsteps after the video content has been gathered.

In the example of FIG. 5, the flowchart 500 ends at module 510 withproviding the video content to a time-synchronized video capturemanagement system. The video content can be provided as appropriate(e.g., via real time streaming, near real time streaming, in batches,etc.).

FIG. 6 shows a flowchart 600 of an example of a method for creating astitched video representation of physical area(s) of interest. It isnoted the flowchart 600 may include a greater or a lesser number ofoperations than those explicitly depicted, and that not all operationsin the flowchart 600 may be necessary for various implementations. Themethod shown in the flowchart 600 may be carried out by atime-synchronized video capture management system.

In the example of FIG. 6, the flowchart 600 starts at module 602 withgathering a time-synchronization trigger. The time-synchronizationtrigger may comprise an event, the occurrence of which results intime-synchronized video capture devices initiating video contentrecording. In some implementations, the time-synchronization triggercomprises specified times and/or specified physical conditions relatingto video capture of a given event. The time synchronization trigger maybe based on sounds, images, motion, or signals from a device (e.g., afob, a beacon, etc.), to name several.

In the example of FIG. 6, the flowchart 600 continues at module 604 withproviding instructions to time-synchronize a plurality oftime-synchronized video capture devices. The time-synchronization mayentail both synchronizing clocks and providing a start time for videofeeds at the plurality of time-synchronized video capture devices.

In the example of FIG. 6, the flowchart 600 continues at module 606 withgathering time-synchronized video content of physical area(s) ofinterest from the time-synchronized video capture devices, thetime-synchronized video content corresponding to fields of view of thephysical area(s) of interest. The time-synchronized video content maycorrespond to field(s) of view of the physical area(s) of interest. Thetime-synchronized video content may be taken by time-synchronized videocapture devices oriented at various perspectives relative to thephysical area(s) of interest.

In the example of FIG. 6, the flowchart 600 continues at module 608 withidentifying an orientation for the fields of view, the orientation beingassociated with a viewer perspective related to the fields of view. Theorientation may be associated with a viewer perspective related to theeach of the fields of view. As examples, the orientation may correspondto a view of a specific side or a specific corner of the physicalarea(s) of interest. The orientation may correspond to a top view, e.g.,of the physical area(s) of interest taken from a drone or from a cameramounted onto a stand. The orientation may correspond to various sideviews or bottom views, e.g., taken from mobile phones, tablet computingdevices, or dedicated video cameras mounted onto relevant mounts.

At an operation 610, the orientations may be marked with orientationmarkers. The orientation markers may comprise data structures that markan orientation of one of a time-synchronized video capture device. Theorientation markers may include information related to the location(global location, relative location relative to the physical area(s) ofinterest, etc.) of the time-synchronized video capture device. Invarious implementations, the orientation markers include Cartesiancoordinates and/or parameters of an axis orthogonal to a referencepoint/plane (e.g., a face, a lens, etc.) of the time-synchronized videocapture device.

At an operation 612, the time-synchronized video content and theorientation markers may be integrated into a three-dimensional domerepresentation of the one or more areas of interest. Thethree-dimensional dome representation may comprise a data structure thatrepresents the cumulative video content captured by multipletime-synchronized video capture devices of the physical area(s) ofinterest at a specific time. The three-dimensional dome representationmay use one or more of the orientation markers to identify orientationsof video content at a specific time and/or to identify relationships ofvideo content taken of a common area of interest with respect to oneanother. In some implementations, the three-dimensional domerepresentation may accommodate different frame rates of video contentfrom different synchronized video capture device.

At an operation 614, a stitched video representation of the area(s) ofinterest may be created using the three-dimensional dome representation.The stitched video representation may comprise a video representation ofthe physical area(s) of interest in which any field of view is visibleat a given time. In some implementations, the stitched videorepresentation includes one or more perspective UI elements that markperspectives associated with each of the field(s) of view. In someimplementations, the perspective UI elements comprise floating virtualobjects (e.g., floating polygons, floating shapes, floating characters,etc.) that reside over portions of the stitched video representationthat correspond to a given perspective. In various implementations, theperspective UI elements allow a reviewer to select perspectives in thestitched video representation at a specific time. A stitched videorepresentation may accommodate different frame rates of video contentfrom different synchronized video capture devices. At an operation 616,the stitched video representation may be provided over acomputer-readable medium to one or more playback devices for display bythe one or more playback devices.

FIG. 7 shows a flowchart 700 of an example of a method for displaying astitched video representation of one or more areas of interest on aplayback device. It is noted the flowchart 700 may include a greater ora lesser number of operations than those explicitly depicted, and thatnot all operations in the flowchart 700 may be necessary for variousimplementations. The method shown in the flowchart 700 may be executedby a playback device.

In the example of FIG. 7, the flowchart 700 starts at module 702 withreceiving a stitched video representation of physical area(s) ofinterest over a computer-readable medium. The stitched videorepresentation may include a plurality of orientations and a perspectiveuser interface (UI) element of visible portion(s). The stitched videorepresentation may include one or more perspective UI elements that markperspectives associated with each of the field(s) of view. In someimplementations, the perspective UI elements comprise floating virtualobjects (e.g., floating polygons, floating shapes, floating characters,etc.) that reside over portions of the stitched video representationthat correspond to a given perspective. In various implementations, theperspective UI elements allow a reviewer to select perspectives in thestitched video representation at a specific time. A stitched videorepresentation may accommodate different frame rates of video contentfrom different time-synchronized video capture device(s). The stitchedvideo representation may be provided to an application on the playbackdevice.

In the example of FIG. 7, the flowchart 700 continues at module 704 withdisplaying the stitched video representation at a first orientation ofthe plurality of orientations. The first orientation may correspond tovideo content taken from a field of view of a time-synchronized videocapture device of the physical area(s) of interest. As a result, thefirst orientation may correspond to video content taken from a firstperspective relative to the physical area(s) of interest.

In the example of FIG. 7, the flowchart 700 continues at module 706 withreceiving a user interaction with the perspective UI element. As anexample, a reviewer's selection of the perspective UI element may bereceived. The perspective UI element may correspond to a secondorientation of the plurality of orientations. At an operation 708, theplayback device(s) may be configured to display the stitched videorepresentation at the second orientation of the plurality oforientations.

Examples Screenshots of a Review Application on a Playback Device

FIG. 8A shows an example of a screenshot 800A of a review application ona playback device. The screenshot 800A includes a depiction of astitched representation of a physical area of interest, which in thisexample, includes a Mixed Martial Arts (MMA) ring. The screenshot 800Aincludes a time-synchronized video capture device control button 802that allows a reviewer to control one or more time-synchronized videocapture devices. FIG. 8B shows an example of a screenshot 800B of areview application on a playback device. The screenshot 800B includes aplayback speed control button 804 that allows a reviewer to control aplayback speed of a stitched video representation of a physical area ofinterest.

FIG. 8C shows an example of a screenshot 800C of a review application ona playback device. The screenshot 800C includes a plurality ofperspective UI elements 806, shown in the figure as floating squaresthat float over the depiction of the MMA ring. Each of the perspectiveUI elements 806 may correspond to a field of view of a time-synchronizedvideo capture device, and of an orientation relative to the MMA ring. Inthis example, the reviewer may select either of the perspective UIelements 806 to choose a perspective to view the MMA ring. Selectingeither of the perspective UI elements 806 may allow the reviewer to seevideo content from a time-synchronized video capture device that isassociated with the perspective UI elements 806. FIG. 8D shows anexample of a screenshot 800D of a review application on a playbackdevice. As shown in FIG. 8D, many perspective UI elements 806 may beimplemented. Many perspectives/orientations may therefore beaccommodated.

FIG. 8E shows an example of a screenshot 800E of a review application ona playback device. In the example of FIG. 8E, a top perspective UIelement has been selected; the perspective has switched to a topperspective of the MMA ring. The reviewer may see video content takenfrom a time-synchronized video capture device positioned above the MMAring.

Example Computer System

FIG. 9 shows an example of a computer system 900, according to someimplementations. The computer system 900 may be a conventional computersystem that may be used as a client computer system, such as a wirelessclient or a workstation, or a server computer system. The computersystem 900 includes a computer 902, I/O devices 904, and a displaydevice 906. The computer 902 includes a processor 909, a communicationsinterface 910, memory 912, display controller 914, non-volatile storage916, and I/O controller 908. The computer 902 may be coupled to orinclude the I/O devices 904 and display device 906.

The computer 902 interfaces to external systems through thecommunications interface 910, which may include a modem or networkinterface. It will be appreciated that the communications interface 910may be considered to be part of the computer system 900 or a part of thecomputer 902. The communications interface 910 may be an analog modem,ISDN modem, cable modem, token ring interface, satellite transmissioninterface (e.g. “direct PC”), or other interfaces for coupling acomputer system to other computer systems.

The processor 909 may be, for example, a conventional microprocessorsuch as an Intel Pentium microprocessor or Motorola power PCmicroprocessor. The memory 912 is coupled to the processor 909 by a bus920. The memory 912 may be Dynamic Random Access Memory (DRAM) and mayalso include Static RAM (SRAM). The bus 920 couples the processor 909 tothe memory 912, also to the non-volatile storage 916, to the displaycontroller 914, and to the I/O controller 908.

The I/O devices 904 may include a keyboard, disk drives, printers, ascanner, and other input and output devices, including a mouse or otherpointing device. The display controller 914 may control in theconventional manner a display on the display device 906, which may be,for example, a cathode ray tube (CRT) or liquid crystal display (LCD).The display controller 914 and the I/O controller 908 may be implementedwith conventional well-known technology.

The non-volatile storage 916 is often a magnetic hard disk, an opticaldisk, or another form of storage for large amounts of data. Some of thisdata is often written, by a direct memory access process, into memory912 during execution of software in the computer 902. One of skill inthe art will immediately recognize that the terms “machine-readablemedium” or “computer-readable medium” includes any type of storagedevice that is accessible by the processor 909 and also encompasses acarrier wave that encodes a data signal.

The computer system 900 is one example of many possible computer systemsthat have different architectures. For example, personal computers basedon an Intel microprocessor often have multiple buses, one of which maybe an I/O bus for the peripherals and one that directly connects theprocessor 909 and the memory 912 (often referred to as a memory bus).The buses are connected together through bridge components that performany necessary translation due to differing bus protocols.

Network computers are another type of computer system that may be usedin conjunction with the teachings provided herein. Network computers donot usually include a hard disk or other mass storage, and theexecutable programs are loaded from a network connection into the memory912 for execution by the processor 909. A Web TV system, which is knownin the art, is also considered to be a computer system, but it may lacksome of the features shown in FIG. 9, such as certain input or outputdevices. A typical computer system will usually include at least aprocessor, memory, and a bus coupling the memory to the processor.

Though FIG. 9 shows an example of the computer system 900, it is notedthat the term “computer system,” as used in this paper, is intended tobe construed broadly. In general, a computer system will include aprocessor, memory, non-volatile storage, and an interface. A typicalcomputer system will usually include at least a processor, memory, and adevice (e.g., a bus) coupling the memory to the processor. The processormay be, for example, a general-purpose central processing unit (CPU),such as a microprocessor, or a special-purpose processor, such as amicrocontroller.

The memory may include, by way of example but not limitation, randomaccess memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM).The memory may be local, remote, or distributed. As used in this paper,the term “computer-readable storage medium” is intended to include onlyphysical media, such as memory. As used in this paper, acomputer-readable medium is intended to include all mediums that arestatutory (e.g., in the United States, under 35 U.S.C. 101), and tospecifically exclude all mediums that are non-statutory in nature to theextent that the exclusion is necessary for a claim that includes thecomputer-readable medium to be valid. Known statutory computer-readablemediums include hardware (e.g., registers, random access memory (RAM),non-volatile (NV) storage, to name a few), but may or may not be limitedto hardware.

The bus may also couple the processor to the non-volatile storage. Thenon-volatile storage is often a magnetic floppy or hard disk, amagnetic-optical disk, an optical disk, a read-only memory (ROM), suchas a CD-ROM, EPROM, or EEPROM, a magnetic or optical card, or anotherform of storage for large amounts of data. Some of this data is oftenwritten, by a direct memory access process, into memory during executionof software on the computer system. The non-volatile storage may belocal, remote, or distributed. The non-volatile storage is optionalbecause systems may be created with all applicable data available inmemory.

Software is typically stored in the non-volatile storage. Indeed, forlarge programs, it may not even be possible to store the entire programin the memory. Nevertheless, it should be understood that for softwareto run, if necessary, it is moved to a computer-readable locationappropriate for processing, and for illustrative purposes, that locationis referred to as the memory in this paper. Even when software is movedto the memory for execution, the processor will typically make use ofhardware registers to store values associated with the software, andlocal cache that, ideally, serves to speed up execution. As used in thispaper, a software program is assumed to be stored at an applicable knownor convenient location (from non-volatile storage to hardware registers)when the software program is referred to as “implemented in acomputer-readable storage medium.” A processor is considered to be“configured to execute a program” when at least one value associatedwith the program is stored in a register readable by the processor.

In one example of operation, the computer system 900 may be controlledby operating system software, which is a software program that includesa file management system, such as a disk operating system. One exampleof operating system software with associated file management systemsoftware is the family of operating systems known as Windows® fromMicrosoft Corporation of Redmond, Wash., and their associated filemanagement systems. Another example of operating system software withits associated file management system software is the Linux operatingsystem and its associated file management system. The file managementsystem is typically stored in the non-volatile storage and causes theprocessor to execute the various acts required by the operating systemto input and output data and to store data in the memory, includingstoring files on the non-volatile storage.

The bus 920 may also couple the processor 909 to the communicationsinterface 910. The communications interface 910 may include one or moreinput and/or output (I/O) devices. The I/O devices may include, by wayof example but not limitation, a keyboard, a mouse or other pointingdevice, disk drives, printers, a scanner, and other I/O devices,including a display device. The display device 906 may include, by wayof example but not limitation, a cathode ray tube (CRT), liquid crystaldisplay (LCD), or some other applicable known or convenient displaydevice. The communications interface 910 may include one or more of amodem or network interface. It will be appreciated that a modem ornetwork interface may be considered to be part of the computer system900. The communications interface 910 may include an analog modem, ISDNmodem, cable modem, token ring interface, satellite transmissioninterface (e.g. “direct PC”), or other interfaces for coupling thecomputer system 900 to other computer systems. The communicationsinterfaces 910 may enable computer systems and other devices to becoupled together in a network.

Several components described in this paper, including clients, servers,and engines, may be compatible with or implemented using a cloud-basedcomputing system. As used in this paper, a cloud-based computing systemis a system that provides computing resources, software, and/orinformation to client devices by maintaining centralized services andresources that the client devices may access over a communicationinterface, such as a network. The cloud-based computing system mayinvolve a subscription for services or use a utility pricing model.Users may access the protocols of the cloud-based computing systemthrough a web browser or other container application located on theirclient device.

This paper describes techniques that those of skill in the art mayimplement in numerous ways. For instance, those of skill in the art mayimplement the techniques described in this paper using a process, anapparatus, a system, a composition of matter, a computer program productembodied on a computer-readable storage medium, and/or a processor, suchas a processor configured to execute instructions stored on and/orprovided by a memory coupled to the processor. Unless stated otherwise,a component such as a processor or a memory described as beingconfigured to perform a task may be implemented as a general componentthat is configured to perform the task at a given time or a specificcomponent that is manufactured to perform the task. As used in thispaper, the term ‘processor’ refers to one or more devices, circuits,and/or processing cores configured to process data, such as computerprogram instructions.

A detailed description of one or more implementations of the inventionis provided in this paper along with accompanying figures thatillustrate the principles of the invention. The invention is describedin connection with such implementations, but the invention is notlimited to any implementation. The scope of the invention is limitedonly by the claims and the invention encompasses numerous alternatives,modifications and equivalents. Numerous specific details are set forthin the following description in order to provide a thoroughunderstanding of the invention. These details are provided for thepurpose of example and the invention may be practiced according to theclaims without some or all of these specific details. For the purpose ofclarity, technical material that is known in the technical fieldsrelated to the invention has not been described in detail so that theinvention is not unnecessarily obscured.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

Techniques described in this paper relate to apparatus for performingthe operations. The apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in acomputer-readable storage medium, such as, but is not limited to,read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, any type of disk including floppydisks, optical disks, CD-ROMs, and magnetic-optical disks, or any typeof media suitable for storing electronic instructions, and each coupledto a computer system bus. Although the foregoing implementations havebeen described in some detail for purposes of clarity of understanding,implementations are not necessarily limited to the details provided.FIG. 9 shows an example of a screenshot of a list of simplified APIs,according to some implementations.

An example of a method developed using techniques described in thispaper, and explored by way of example with reference to FIGS. 10-17includes:

measuring orientation of at least a first, a second and a third cameraof the plurality of cameras;

using the measured orientation to determine a direction of view of theat least first, second and third camera of the plurality of cameras;

making a first iteration of the relative positions of the at leastfirst, second and third camera of the plurality of cameras based on thedetermined direction;

measuring relative distances between the at least first, second andthird camera of the plurality of cameras;

making a second iteration of the relative positions of the at leastfirst, second and third camera of the plurality of cameras based on themeasured relative distances; and

using the first iteration and the second iteration to determine therelative positions of the at least first, second and third camera of theplurality of cameras with respect to each other.

An example of a system developed using techniques described in thispaper, and explored by way of example with reference to FIGS. 10-17,includes:

a set of sensors operable to measure orientation of at least a first, asecond and a third camera of the plurality of cameras, wherein thesensors of the set of sensors is installed in the at least first, secondand third camera of the plurality of cameras; and

a server configured to:

determine a direction of view of the at least first, second and thirdcamera of the plurality of cameras using the measured orientation;

make a first iteration of relative positions of the at least first,second and third camera of the plurality of cameras based on thedetermined direction;

measure relative distances between the at least first, second and thirdcamera of the plurality of cameras;

make a second iteration of relative positions of the at least first,second and third camera of the plurality of cameras based on themeasured relative distances; and

determine the relative positions of the at least first, second and thirdcamera of the plurality of cameras with respect to each other using thefirst iteration and the second iteration.

The plurality of cameras are configured to record multiple videos of anobject from different directions and positions within a location. Suchvideos may, for example, be provided to a user on a user device havingan application installed therein, which is further operable to providethe user an option for selecting and playing at least one recorded videothereon. Increasing the number of cameras increases the number of views.The number of cameras can be a few cameras (e.g., 3), tens of cameras(e.g., 50), hundreds of cameras (e.g., 500) or even thousands of cameras(e.g., 10,000). In a specific implementation, the cameras are videocameras that can take still images as well. In a specificimplementation, the plurality of cameras includes a portable electronicdevice, such as a smart phone with a camera, a phone with a camera, aniPod, an iPad, a smart camera, such as a digital camera, or the like. Ina specific implementation, the cameras comprise sensors of the set ofsensors for measuring an orientation of the cameras. The set of sensorscan include by way of example but not necessarily by limitation, amagnetometer, gyroscope, and accelerometer.

The plurality of cameras record videos (and/or still images) related toan event taking place within a location. The location can be an openfield, an indoor location, or some other location. The event may be asports event (for example, a basketball match, a badminton match, abaseball match, a football match, a race and the like), theatricalperformances (for example opera, dramatics and the like), liveperformances (for example, musical band performances, dance performanceand/or competition, singing performance and/or competition and awardpresentations), or another applicable venue.

The plurality of cameras can include cameras used for recording (orbroadcasting), for example, a sports event. The plurality of cameras canbe used, for example, in a training situation, such as professionalsports training situation, a yoga instruction training situation, orsome other training situation in which visual instruction is useful. Asanother example, the plurality of cameras may relate to a monitoringpurpose of moving objects, such as people within an enclosure, animalswithin a zoo or nature preserve, or other venues in which monitoringmoving objects is useful.

For example, the object can be players involved in the sports event.Similarly, an object can be a participant of a bike race, a skier, agolfer, a baseball player, a gymnast, or participant in some othervenue. Thus, the system and method can be useful for bicycle ridingtraining, ski training, golf swing training, baseball batting training,gymnastics training, etc. Further, it may be evident to those skilled inthe art that the object should be captured (or recorded) by at leastmore than one camera (of the plurality of cameras) simultaneously forproviding at least more than one view of the object to the user.

In a specific implementation, at least a subplurality of the pluralityof cameras are arranged in the location where the event is taking place,such as, in a football stadium for recording videos of the footballmatch. The subplurality of cameras may be configured in the location byattaching the subplurality of cameras to a frame. The frame may beconfigured to have various shapes (uniform or arbitrary shape) based ona need of a setup. For example, the frame may be configured to have adome, a sphere, a hemisphere, a cylinder, an oval, a circular, apolygonal, or some other applicable shape. In a specific implementation,the subplurality of cameras is associated with a user (for example, aspectator, a designated camera person, a stadium staff and the like) whois recording videos of the object.

In a specific implementation, cameras comprise a communication medium, auser interface (such as, display, touch screen, buttons), camera optics(for taking still images and recording video), a microphone, a battery,a location means (such as, GPS sensor) for detecting location of the atleast first, second and third camera of the plurality of cameras. In aspecific implementation, the cameras are communicably coupled to aserver via the communication medium, which can be a wired, wireless, ora combination thereof. Examples of the communication medium may includebut not be limited to Bluetooth, Wireless LANs (WLANs), Wireless WANs(WWANs), Wireless MANs (WMANs), the Internet, second generation (2G)telecommunication networks, third generation (3G) telecommunicationnetworks, fourth generation (4G) telecommunication networks, andWorldwide Interoperability for Microwave Access (WiMAX) networks.

In a specific implementation, a user device is configured to displayvideo recorded by the plurality of cameras, based on the user selection.According to an embodiment, there may be two user devices, such as afirst user device and a second user device. The first user device isassociated with a user watching a recorded video and the second userdevice is associated with an administrator of the system. In anembodiment, the administrator controls displaying of the videos on thefirst user device. Specifically, the administrator segregates somevideos from the recorded videos based on quality, viewer's demand andthe like, and presents the segregated videos first user devicethereafter. The user device can also be communicably coupled to theserver.

In a specific implementation, the server is configured to receiveinformation (i.e. videos or images) from cameras and process thereafterbefore sending the information to the user device. For example, theserver is configured to simultaneously receive all the videos (havingdifferent camera views) recorded by the plurality of cameras. The servermay be communicably coupled to a datastore for storing the information(i.e. videos or images) received from the plurality of cameras.Alternatively, the datastore may store metadata of each of the pluralityof cameras, such as model name, model number and the like.Alternatively, the system could also be established such that no serveris needed, i.e. that each camera of the plurality of cameras can performthe function of the server.

In a specific implementation, an application is installed in a userdevice that is operable for providing the option to the user forselecting and viewing a video from the multiple videos recorded by theplurality of cameras. For example, a user interface of the applicationmay automatically present a mesh of graphical elements, which correspondto the plurality of cameras arranged to record the videos. The user canselect a particular graphical element from the mesh of graphicalelements for viewing the videos recorded by the camera associatedtherewith. The automatic presentation of mesh of such graphical elementson the user device is possible when the relative positions of theplurality of cameras with respect to each other are known. For example,the relative positions may be determined based on a calibration method,which is described herein.

In a specific implementation, measuring the orientation of the at leastfirst, second, and third camera of the plurality of cameras is performedby a set of sensors. For example, the set of sensors may be configuredto provide orientation information of the at least first, second, andthird camera of the plurality of cameras to the server. In anembodiment, the orientation information comprises data about position ofthe at least first, second, and third camera of the plurality of cameraswith respect to a reference plane. Further, the orientation informationcomprises angles at which the plurality of cameras is arranged withrespect to X axis of a coordinate system. For example, the orientationinformation includes values of angle alpha (α) (formed with respect to Xaxis), and angle beta (β) (formed between a surface defined by X and Yaxis).

In a specific implementation, the orientation information is measuredbased on visual analysis. In another implementation, the location means(such as, GPS sensor) installed in the at least first, second, and thirdcamera of the plurality of cameras may detect location of the at leastfirst, second and third camera of the plurality of cameras. In anotherimplementation, the magnetometer behaves as magnetic compass and isoperable to find an orientation (or a likely/approximate position) ofeach of the plurality of cameras, with respect to magnetic field of thelocation. In another implementation, the gyroscope is operable to findan orientation of each of the plurality of cameras based on earth'sgravity and the accelerometer is operable to measure non-gravitationalacceleration. Thereafter, a direction of view of the at least first,second, and third camera of the plurality of cameras is determined usingthe measured orientation. For example, the server, which is communicablycoupled to the plurality of cameras, is configured to receive theorientation information from the set of sensors (installed in theplurality of cameras) and determine the direction of view of the atleast first, second, and third camera of the plurality of camerasthereafter.

Alternatively, the direction of view of the at least first, second andthird of the plurality of cameras is determined using the measuredorientation and/or information associated with a defined geometricalshape of the location where the plurality of cameras is arranged. Insuch instance, the database of the server has pre-stored informationassociated with the geometrical shape of the location. In an example,dimensions of the location (such as, a badminton court of cuboidalshape, having a length of 200 meters, a width of 100 meters and a height150 meters) may be stored therein. Similarly, dimensions of the location(such as, a track field of a sphere or a quasi-sphere shape having aradius of 100 meters) may also be stored therein. The first iteration ofthe relative positions of the at least first, second and third camera ofthe plurality of cameras is made thereafter based on the determineddirection of view. Specifically, the server uses the determineddirection as a first approximation to find the relative positions of theat least first, second and third camera of the plurality of cameras,i.e. to determine a likely position of the plurality of cameras assumingthat these are equidistant from the object. Further, the first iterationis based on assuming the orientation of the at least first, second andthird camera of the plurality of cameras is towards an object.

In a specific implementation, a server is operable to make the firstiteration of the relative positions of the at least first, second, andthird camera of the plurality of cameras based on calculating theposition by looking on a value in respect to a common co-ordinate baseof the plurality of cameras and the orientation information (such as,information measured by magnetometer, using a magnetic compass).Typically, the magnetic compass has a measuring circle where 0 degreecorresponds to North, 90 degrees correspond to East, 180 degreescorrespond to South and 270 degrees correspond to West. In an example,if the first camera is aiming to direction α=270 degrees, based on theassumption that the orientation of the at least first, second and thirdcamera is towards an object, thus its position around the object is in a180=90 degrees.

Relative distances between the at least first, second, and third cameraof the plurality of cameras can be measured thereafter. For example, therelative distances between the plurality of cameras are measured by theserver. In an embodiment, as the first iteration of the relativepositions of the at least first, second and third of the plurality ofcameras is made based on an assumption that each of the plurality ofcameras point towards the object, and may not always yield correctresults. For example, if the cameras are arranged in a way that thesecond camera and the third camera of the plurality of cameras point ina direction away from the object, then in such instance, the firstiteration of the relative positions of the at least first, second andthird camera of the plurality of cameras will not hold good. Also, ifthe cameras are arranged adjacent to each other (for example, on arectangular track) and in a way that the second and the third cameraspoint in a same direction, then in such instance, the second and thethird cameras could not be distinguished and only the determineddirectional information would not provide sufficient information toorder these cameras since these point to the same direction. This may beaddressed by measuring the relative distances between the at leastfirst, second and third camera of the plurality of cameras.

In an embodiment, the measured relative distances may be based oncalculating received signal strength indicator (RSSI) values between theat least first, second and third camera of the plurality of cameras.Specifically, each of the plurality of cameras may be configured to sendand receive radio waves using the communication medium, such asBluetooth (BT) of each of the plurality of cameras. For example, thefirst camera is configured to send pings over Bluetooth and the secondand the third cameras are configured to receive the pings and measureRSSI thereof. Similarly, the second camera is configured to send pingsover Bluetooth and the first and the third cameras are configured toreceive the pings and measure RSSI thereof. Similarly, the third camerais configured to send pings over Bluetooth and the first and the secondcameras are configured to receive the pings and measure RSSI thereof.Additionally or alternatively the relative distance between cameras canbe determined with Global Positioning System (GPS). Additionally oralternatively the relative distance between cameras can be determinedwith voice. In this alternative embodiment one or more of the camerasmake a sound and other cameras (the recording cameras) record the sound.The arrival time of the sound to the recording camera is used todetermine the relative position of the cameras. The recording camerasmay additionally use the moment of sending, i.e. the time of travel ofthe voice, as well as optionally a triangulation technique with otherrecording cameras to determine their relative positions. Additionally oralternatively other radio technologies than BT can be used, such as (butnot limited to) Wireless Local Area Network, low power BT etc.

The second iteration of the relative positions of the at least first,second, and third camera of the plurality of cameras is made thereafter,based on the measured relative distances. The second iteration of therelative positions is based on comparing the measured relative distancesto theoretical distance between the at least first, second and thirdcamera of the plurality of cameras. For example, the second iteration ofthe relative positions may be based on dividing the measured relativedistances and the theoretical distance between the at least first,second and third camera of the plurality of cameras. In an embodiment,the theoretical distance defines a geometrical shape of the location forexample, circular, rectangular and the like. Further, the relativepositions of the at least first, second, and third camera of theplurality of cameras with respect to each other are determined using thefirst iteration and the second iteration.

In an example, if distances between the relative positions of each ofthe at least first, second, and third camera of the plurality of cameras(after the second iteration) comes out to be substantially same (withinerror margins), then the orientation information and the relativepositions of each of the plurality of cameras is right. Otherwise, auser (such as, an administrator or a user associated with a camera ofthe plurality of cameras) is instructed to correct position of the atleast first, second and third camera of the plurality of cameras.

In a specific implementation, determined relative positions of a first,second, and third camera of the plurality of cameras is shown in a userdevice. If a user is not satisfied with the positions, then graphicalelements can be dragged and dropped to appropriate relative positionsusing the user interface of the user device. Alternatively, if the useragrees with the positions, then the videos of the object can be recordedusing the plurality of cameras thereafter. For example, the user sends astart recording command substantially simultaneously to each of theplurality of cameras and thereafter receives recorded videos (or in realtime receives and stores the streamed videos from the plurality ofcameras). In a specific implementation, each of the plurality of camerascomprises an application installed therein which is operable to startrecording (and storing or streaming content to target device/server) thevideo substantially at the same time as other cameras. In a specificimplementation, a server has a file or a set of files for reproducingvideos from multiple angles.

In a specific implementation, if a user starts a new recording sessionand wants to position the plurality of cameras the way the cameras werepositioned earlier, the user loads the earlier used camera configurationfor reference and the user interface indicates the current camerapositions. The user can now physically move the cameras to correctpositions to match with the earlier used configuration. The userinterface may be configured to display indicators that show when eachcamera position matches with the earlier used position.

Referring to FIG. 10, illustrated is a schematic illustration of anenvironment 1000. The environment 1000 includes cameras 1002 a, 1002 band 1002 c (hereinafter collectively referred to as the plurality ofcameras 1002). The plurality of cameras 1002 are arranged in such a waythat the plurality of cameras 1002 hemispherically surrounds an object1004 for recording videos of the object 1004. Specifically, the cameras1002 a, 1002 b and 1002 c point to directions 1006 a, 1006 b and 1006 c(hereinafter collectively referred to as directions 1006) respectivelyfor recording the videos of the object 1004. Each of the plurality ofcameras 1002 is associated with a respective direction, i.e. each of theplurality of cameras 1002 has a particular direction of view, whichenables each of the plurality of cameras 1002 to record videos of theobject 1004 from a certain angle and direction. The directions 1006 aredetermined based on orientation information of the cameras 1002 a, 1002b and 1002 c, measured by orientation sensors (1008 a, 1008 b and 1008c) respectively, installed therein, described in detail with referenceto, for example, FIG. 15 and FIG. 16.

FIG. 10 is intended to illustrate that each of the plurality of cameras1002 is communicably coupled to a plurality of user devices 1010 a and1010 b (hereinafter collectively referred to as plurality of userdevices 1010) via a communication medium 1012. For example, each of theplurality of cameras 1002 can host an application, which is configuredto connect each of the plurality of cameras 1002 to each of theplurality of user devices 1010 via the communication medium 1012.

FIG. 10 is intended to illustrate a server 1014 that is communicablycoupled to each of the plurality of cameras 1002 and each of theplurality of user devices 1010 via the communication medium 1012. Thevideos of the object 1004 recorded by each of the plurality of cameras1002 may be stored in a datastore (not shown), which is communicablycoupled to the server 1014 along with other related metadata of each ofthe plurality of cameras 1002.

In a specific implementation, the server 1014 is operable to process thereceived videos from the plurality of cameras 1002 and furtherconfigured to send the processed videos to the plurality of user devices1010. A user interface (not shown) of each of the plurality of userdevices 1010 is configured to enable a user to change direction of viewof the plurality of cameras 1002 depending upon the video received fromthe plurality of cameras 1002. The direction of view (or recordingdirections i.e. corresponding viewing positions) of the plurality ofcameras 1002 can be changed based on determined directions (described insubsequent figures).

The plurality of user devices 1010 is configured to remotely control theplurality of cameras 1002. For example, the user interface of each ofthe plurality of user devices 1010 can be configured to remotely controlthe plurality of cameras 1002, e.g., by sending commands to start/stoprecording or streaming image/video of the object 1004.

Referring to FIGS. 11-12, illustrated are example illustrationsdepicting orientation arrangements, such as orientation arrangements1100 and 1200, respectively, of the plurality of cameras 1002 around theobject 1004.

As shown in FIG. 11, cameras 1102 a, 1102 b and 1102 c are arrangedaround the object 1004 in a circular fashion forming the orientationarrangement 1100. The cameras 1102 a, 1102 b and 1102 c have orientationsensors 1104 a, 1104 b and 1104 c respectively, installed therein.

The orientation sensors 1104 a, 1104 b and 1104 c are configured tomeasure orientation data of the cameras 1102 a, 1102 b and 1102 crespectively and further configured to send the measured orientationdata to the server 1014 (shown in FIG. 10). Specifically, the server1014 is configured to process the orientation data of the cameras 1102a, 1102 b and 1102 c and further configured to determine a direction ofview of the cameras 1102 a, 1102 b and 1102 c using the measuredorientation, described in detail in FIGS. 15 and 16. The server 1014 isfurther operable to make a first iteration of the relative positions ofthe cameras 1102 a, 1102 b and 1102 c based on the determined direction,assuming that the cameras 1102 a, 1102 b and 1102 c are placed at equaldistance from the object 1004.

Specifically, the server 1014 is configured to determine the likelypositions of the cameras 1102 a, 1102 b and 1102 c by considering valuein respect to a common co-ordinate base of the cameras 1102 a, 1102 band 1102 c. For example, the camera 1102 a has a direction, α=270degrees, thus its position around the object 1004 is estimated to beα−180=90 degrees. Further, this estimation determines a relativeposition of the cameras 1102 a, 1102 b and 1102 c.

As shown in FIG. 12, the cameras 1102 a, 1102 b and 1102 c are arrangedaround the object 1004 in a circular fashion forming the orientationarrangement 1200. Specifically, the orientation arrangement 1200includes the cameras 1102 a, 1102 b and 1102 c having sensors 1104 a,1104 b and 1104 c, respectively, and arranged in a counter clockwisemanner. Further, the cameras 1102 a, 1102 b and 1102 c are arranged in away that the direction of views are determined by the sensors 1104 a,1104 b and 1104 c, and shown as directions 1202, 1204 and 1206respectively. In such instance, the server 1014 is configured to measurerelative distances between the cameras 1102 a, 1102 b and 1102 c andfurther configured to make a second iteration of the relative positionsof the cameras 1102 a, 1102 b and 1102 c based on the measured relativedistances, which is described in detail in FIG. 13.

Referring now to FIG. 13, illustrated is an example illustration of anarrangement 1300 depicting positions of the cameras 1102 a, 1102 b and1102 c illustrated in the orientation arrangements 1100 and 1200. FIG.13 illustrates measurement of relative distances between the cameras1102 a, 1102 b and 1102 c with respect to each other. Specifically, therelative distances are measured based on calculating received signalstrength indicator values between the cameras 1102 a, 1102 b and 1102 c.As shown, the relative distances between the cameras 1102 a, 1102 b and1102 c are dAB, dAC and dBC.

Further, the second iteration of the relative positions of the cameras1102 a, 1102 b and 1102 c is made based on the measured relativedistances. Specifically, the second iteration of the relative positionsis based on comparing the measured relative distances and a theoreticaldistance between the cameras 1102 a, 1102 b and 1102 c. As shown, thetheoretical distance between the cameras 1102 a, 1102 b and 1102 c aredtAB, dtAC and dtBC. More specifically, the second iteration of therelative positions is based on dividing the measured relative distancesand the theoretical distance between the cameras 1102 a, 1102 b and 1102c, illustrated below as:

rAB=dAB/dtAB,

rAC=dAC/dtAC,

rBC=dBC/dtBC.

The values rAB, rAC and rBC are the relative positions of the cameraswith respect to each other. If distances between the relative positionsof the cameras 1102 a, 1102 b and 1102 c (after the second iteration)come out to be substantially same, then the orientation information andthe relative positions of the cameras 1102 a, 1102 b and 1102 c areright. Otherwise, a user (such as, an administrator or a user associatedwith a camera of the cameras 1102 a, 1102 b and 1102 c) may beinstructed to correct position of the cameras 1102 a, 1102 b and 1102 c.

Referring now to FIG. 14, illustrated is an example illustration 1400for determining relative positions of a plurality of cameras, such ascameras 1402, 1404, 1406, 1408 and 1410 with respect to each other. Asshown, the cameras 1402, 1404, 1406, 1408 and 1410 are arranged aroundan object 1412 (unlike circular manner, as shown in FIG. 13). Each ofthe cameras 1402, 1404, 1406, 1408 and 1410 is communicably coupled to auser device 1414 via the communication medium 1012 (as shown in FIG.10). Specifically, each of the cameras 1402, 1404, 1406, 1408 and 1410hosts an application, which is configured to connect each of the cameras1402, 1404, 1406, 1408 and 1410 to the user device 1414 via thecommunication medium 1012 (shown in FIG. 10).

Further, each of the cameras 1402, 1404, 1406, 1408 and 1410 hasorientation sensors 1416, 1418, 1420, 1422 and 1424, respectively,installed therein. Each of the orientation sensors 1416, 1418, 1420,1422 and 1424 are configured to measure an orientation data of thecameras 1402, 1404, 1406, 1408 and 1410, respectively, and furtherconfigured to communicate the measured orientation data to the userdevice 1414. The server 1014 (as shown in FIG. 10) or the user device1414 is configured to use the orientation data to determine direction ofview of each of the cameras 1402, 1404, 1406, 1408 and 1410 from wherethe cameras 1402, 1404, 1406, 1408 and 1410 are recording videos of theobject 1412. As shown, based on the determined direction of views, it isclear that the cameras 1402, 1404 and 1406 point in a same direction andcan distinguish between the cameras 1402, 1404 and 1406, and the cameras1408 and 1410.

Further, the server 1014 (as shown in FIG. 10) or the user device 1414is configured to determine the relative positions of the cameras 1402,1404, 1406, 1408 and 1410 with respect to each other based oncalculating the received signal strength indicator values between eachof the cameras 1402, 1404, 1406, 1408 and 1410 (and optionally betweenthe cameras 1402, 1404, 1406, 1408 and 1410 and the user device 1414).In an instance, the relative distance between the cameras 1402 and 1410is greater than the relative distance between the cameras 1404 and 1410,which is further greater than the relative distance between the cameras1406 and 1410. This enables the server 1014 (as shown in FIG. 10) or theuser device 1414 to determine an order in which the cameras 1402, 1404,1406, 1408 and 1410 are arranged.

Moreover, the user device 1414 is configured to display the determinedrelative positions of the cameras 1402, 1404, 1406, 1408 and 1410 withrespect to each other to a user. A user interface (not shown) of theuser device 1414 comprises a mesh of graphical elements, correspondingto each of the cameras 1402, 1404, 1406, 1408 and 1410, which allows theuser to change the relative positions of the cameras 1402, 1404, 1406,1408 and 1410 (if needed) by dragging and dropping the graphicalelements to appropriate/desired relative positions.

The user device 1414 is further configured to send commands for startingrecording/streaming of the object 1412 by the cameras 1402, 1404, 1406,1408 and 1410 and further configured to receive the recorded content,such as audios/videos and store thereafter in a database (not shown)communicably coupled thereto or with a server 1014 (of FIG. 10).Furthermore, the user device 1414 is configured to display, via the userinterface, the recorded video to the user.

Referring to FIG. 15, illustrated is a schematic illustration of aco-ordinate system 1500 (designated with X, Y Z axes) used forpracticing various embodiments of the present disclosure. As shown, thecoordinate system 1500 includes a camera 1502 placed at a location 1504,which is represented by coordinates: (x0, y0, z0). Further, a directionof view of the camera 1502 is indicated with an arrow 1506. Thedirection of view of the camera 1502 is estimated from orientationinformation of the camera 1502, measured by an orientation sensor 1508installed therein. Specifically, the direction of view is represented inangle alpha (α) 1510 from the X axis and in angle beta (β) 1512 fromsurface defined by the X and Y axes.

Referring now to FIG. 16, illustrated is an example illustrationdepicting a plurality of cameras in a coordinate system 1600. As shown,the coordinate system 1600 includes the cameras 1602, 1604 and 1606,each having angle (beta β=0, as shown in FIG. 15) as seen from adirection of Z-axis (not shown). Also, shown is an object 1608 to berecorded by the 1602, 1604 and 1606. Specifically, FIG. 16 illustratesorientation information of the cameras 1602, 1604 and 1606, determinedby orientation sensors 1610, 1612 and 1614 installed thereinrespectively.

The orientation information includes angles at which the cameras 1602,1604 and 1606 are arranged with respect to X axis, which is used todetermine a direction of view of each of the cameras 1602, 1604 and 1606and their relative position thereafter. In an instance, the orientationsensor 1610 installed in the camera 1602 determines that the camera 1602is in angle α1 with respect to X-axis. This data may be used by the userdevice 1414 (shown in FIG. 14) or by the server 1014 (shown in FIG. 10)to determine a direction of view of the camera 1602, which is indicatedwith an arrow 1616, and a position of the camera 1602, indicated at alocation 1618 represented by coordinates (x1, y1).

Further, the orientation sensor 1612 installed in the camera 1604determines that the camera 1604 is in angle α2 with respect to X-axis.This data may be used by the user device 1414 (of FIG. 14) or by theserver 1014 (of FIG. 10) to determine a direction of view of the camera1604, which is indicated with an arrow 1620 and a position of the camera1604, indicated at a location 1622 represented by coordinates (x2, y2).The orientation sensor 1614 installed in the camera 1606 determines thatthe camera 1606 is in angle α3 with respect to X-axis. Similarly, thisdata may be used by the user device 1414 (of FIG. 14) or by the server1014 (of FIG. 10) to determine a viewing direction of the camera 1606,which is indicated with an arrow 1624 and a position of the camera 1606,indicated at a location 1626 represented by coordinates (x3, y3).Further, the object 1608 is placed at a location 1628 represented bycoordinates (x0, y0). Embodiment of the disclosure is not limited to twodimensional case i.e. cameras 1604, 1606, 1608 can be in a same level inrespect to a XY plane (z1=z2=z3) or the cameras can be in differentplanes in respect to a XY plane (z1< >z2, z2< >z3 for example).

FIG. 17 depicts a flowchart 1700 of an example of a method fordetermining relative positions of the plurality of cameras with respectto each other within a location. Those skilled in the art wouldrecognize that the flowchart 1700 can be used in association with thesystem 1000, explained in conjunction with the FIG. 10.

In the example of FIG. 17, the flowchart 1700 starts at module 1702,where orientation of at least a first, a second and a third camera ofthe plurality of cameras are measured. Examples of how to measureorientations are described with reference to FIGS. 10-16.

The flowchart 1700 continues to module 1704, where a direction of viewof the at least first, second and third camera of the plurality ofcameras is determined using the measured orientation. Examples of how todetermine a direction of view at the cameras are described withreference to FIGS. 10-16.

The flowchart 1700 continues to module 1706, where a first iteration ofthe relative positions of the at least first, second and third camera ofthe plurality of cameras is made based on the determined direction.Examples of how to make a first iteration of relative positions aredescribed with reference to FIGS. 10-16.

The flowchart 1700 continues to module 1708, where relative distancesbetween the at least first, second and third camera of the plurality ofcameras are measured. Examples of how to measure relative distancesbetween cameras are described with reference to FIGS. 10-16.

The flowchart 1700 continues to module 1710, where a second iteration ofthe relative positions of the at least first, second and third camera ofthe plurality of cameras is made based on the measured relativedistances. Examples of how to make a second iteration of relativepositions are described with reference to FIGS. 10-16.

The flowchart 1700 continues to module 1712, where the relativepositions of the at least first, second and third camera of theplurality of cameras with respect to each other are determined using thefirst iteration and the second iteration. Examples of how to determinerelative positions of cameras are described with reference to FIGS.10-16.

The modules 1702 to 1712 are only illustrative and other alternativescan also be provided where one or more modules are added, one or moremodules are removed, or one or more modules are provided in a differentsequence without departing from the scope of the claims herein. Forexample, in the flowchart 1700, the first iteration is based on assumingthe orientation of the at least first, second and third camera of theplurality of cameras is towards an object. Further, in the flowchart1700, the measured relative distances are based on calculating receivedsignal strength indicator values between the at least first, second andthird camera of the plurality of cameras. Furthermore, in the flowchart1700, the second iteration of the relative positions is based oncomparing the measured relative distances and a theoretical distancebetween the at least first, second and third camera of the plurality ofcameras. Moreover, in the flowchart 1700, the theoretical distance isbased on assuming the location to be of a defined geometric shape.

Several components described in this paper, including clients, servers,and engines, may be compatible with or implemented using a cloud-basedcomputing system. As used in this paper, a cloud-based computing systemis a system that provides computing resources, software, and/orinformation to client devices by maintaining centralized services andresources that the client devices may access over a communicationinterface, such as a network. The cloud-based computing system mayinvolve a subscription for services or use a utility pricing model.Users may access the protocols of the cloud-based computing systemthrough a web browser or other container application located on theirclient device.

This paper describes techniques that those of skill in the art mayimplement in numerous ways. For instance, those of skill in the art mayimplement the techniques described in this paper using a process, anapparatus, a system, a composition of matter, a computer program productembodied on a computer-readable storage medium, and/or a processor, suchas a processor configured to execute instructions stored on and/orprovided by a memory coupled to the processor. Unless stated otherwise,a component such as a processor or a memory described as beingconfigured to perform a task may be implemented as a general componentthat is configured to perform the task at a given time or a specificcomponent that is manufactured to perform the task. As used in thispaper, the term ‘processor’ refers to one or more devices, circuits,and/or processing cores configured to process data, such as computerprogram instructions.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

Techniques described in this paper relate to apparatus for performingthe operations. The apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in acomputer-readable storage medium, such as, but is not limited to,read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, any type of disk including floppydisks, optical disks, CD-ROMs, and magnetic-optical disks, or any typeof media suitable for storing electronic instructions, and each coupledto a computer system bus. Although the foregoing implementations havebeen described in some detail for purposes of clarity of understanding,implementations are not necessarily limited to the details provided.FIG. 9 shows an example of a screenshot of a list of simplified APIs,according to some implementations.

1. A system comprising: a time-synchronized video capture devicemanagement engine configured to gather time-synchronized video contentof one or more areas of interest from time-synchronized video capturedevices, the time-synchronized video content corresponding to fields ofview of the one or more areas of interest; a three-dimensional domerepresentation integration engine coupled to the time-synchronized videocapture device management engine, the three-dimensional domerepresentation integration engine configured to: identify an orientationfor each of the fields of view, the orientation being associated with aviewer perspective related to the each of the fields of view; mark theorientations with orientation markers; integrate the time-synchronizedvideo content and the orientation markers into a three-dimensional domerepresentation of the one or more areas of interest, thethree-dimensional dome representation configured to arrange thetime-synchronized video content in accordance with the orientationmarkers; a stitched video representation management engine coupled tothe three-dimensional dome representation integration engine, thestitched video representation management engine configured to create astitched video representation of the one or more areas of interest usingthe three-dimensional dome representation, the stitched videorepresentation configured to facilitate display of any of thetime-synchronized video content at a specific time, and the stitchedvideo representation using the orientation markers to facilitateswitching between the time-synchronized video content at the specifictime; a playback device management engine coupled to the stitched videorepresentation management engine, the playback device management engineconfigured to provide the stitched video representation to one or moreplayback devices for display by the one or more playback devices.